Coming soon
Message queues that push.
The Queue Expert is a managed message queue, like Amazon SQS. The difference: your code keeps a WebSocket open, and each message is sent to it the moment it arrives. No polling. No receive calls that come back empty.
The queue doubles as a load distributor, not a load balancer. Instead of spreading messages evenly, it gives each one to the first subscriber with room. Subscribers you don't need stay idle, so you can remove that unused capacity.
What it does
Everything in this list works in our queue engine today.
Delivery
-
Pushed, not polled
Subscribers keep one WebSocket open. Each message is sent to a subscriber as soon as it arrives. No polling, and no receive calls that come back empty.
-
Delivered in publish order
Messages go out in the order they were published. A delayed message joins the back of the line when its delay ends.
-
A load distributor that shows spare capacity
Each new message goes to the first subscriber in line that has room, not round robin. So the subscribers at the front do the work, and the ones at the back stay idle. Idle subscribers are capacity you can remove.
-
Each subscriber sets its pace
A subscriber says how many unacknowledged messages it can hold, and can change that number while it is connected. The queue sends it no more until it acknowledges some.
-
No visibility timeout to tune
A message stays with one subscriber until that subscriber acknowledges or rejects it. If the subscriber logs out, or goes away and does not come back in time, its messages go to the back of the queue for other subscribers.
-
Delays of up to two days
Publish a message now, and have it delivered later: up to two days later.
-
Fan-out too
Every queue comes with a copy queue. It sends each message to every connected subscriber at once, like a topic in Amazon SNS, and tells the publisher how many subscribers got it. It keeps no messages, so it suits live updates.
-
Log in and publish in one step
A publisher can put a message in its login request. So a short job can connect and publish with one request and one reply.
Reliability
-
Durability for each message
The publisher chooses. None: kept in memory only. Eventual: written to disk. Immediate: on disk before the publish is confirmed.
-
Sessions that survive
If a connection drops, a subscriber that has received a message can resume its session within a time you choose, from 10 seconds to a day. It keeps the messages it held, and gets back its place in the line of subscribers. Sessions are kept on disk, so they also survive a restart or a crash of the queue.
-
Replay after a reconnect
When it resumes, a subscriber can ask for the messages it was sent but has not acknowledged to be sent again, in case they were lost with the old connection.
-
Dead-letter queue
A subscriber can reject a message it cannot handle. The message moves to the queue's dead-letter queue, where an admin can read it or delete it.
-
Your own key, to spot duplicates
Give a message your own key, such as
order-1234. The subscriber gets the key with the message, so it can tell when a message was published twice. -
Checksums, and a safe stop
Every message on disk has a checksum. A message that the disk damaged is found when the queue starts, and moved to the dead-letter queue instead of being delivered. If the disk fails while the queue runs, the queue stops, rather than carry on with a broken disk.
Seeing and running your queue
-
Watch your queue
Admins can get live statistics as often as they choose: waiting, unacknowledged, delayed, dead-lettered and stored messages, and the publishers, subscribers and saved sessions. The queue keeps a history of these, and of its network and disk use.
-
Manage your messages
Admins can page through the waiting messages, add and delete messages, move a message to the dead-letter queue, and read or delete the messages there.
-
Change settings while it runs
Admins can change settings without a restart: how long a saved session waits, the biggest message, the most a subscriber may hold, and how often and how long statistics and usage are kept. Changes are saved, so they survive a restart.
Connections and security
-
A login for each role
Publishers, subscribers and admins each log in with their own username and password. Passwords are kept only as Argon2 hashes.
-
Secure, standard WebSockets
Every connection uses TLS 1.2 or newer. Messages can be compressed (permessage-deflate).
-
Text or binary
A message body can be text, or binary data sent as base64. Each message gets an id that sorts in time order, and the time it arrived.
-
Match replies to requests
Any request can carry a context: a number, or text of up to 256 bytes, such as your own request number. The reply carries it back, so your code knows which request it answers.
Your queue, on its own machine
Each queue runs on a virtual machine of its own. No other customer's queue or traffic shares it. Every machine has a public IPv6 address, and a public IPv4 address if you want one.
-
Dev/test
One small machine, to build and test your code against.
-
Production
One machine for each queue, in three sizes: small, medium and large.
-
Production with Replication Planned
The same sizes, with two machines for each queue.
Register your interest
Leave your email address, and tell us which AWS region is closest to you. We will email you when The Queue Expert opens.
Your privacy. We keep your email address, the region you chose, and the time you sent the form. We use them only to tell you when we open, and to learn which regions to open first. We do not share them or sell them, and this page sets no cookies. If you use the light/dark switch, your browser remembers your choice on your device only. Our server keeps your IP address in memory for about an hour, to stop spam, and does not store it. We also count visits to this page, without a cookie. To count each visitor once a day, our server makes codes from your IP address and browser name with a secret that changes every day. It keeps the codes in memory until the day ends (UTC), and stores only the day's totals and the names of the websites that visitors came from.