Valkey transactions and Lua scripts: MULTI, WATCH, EVAL

Valkey runs commands one at a time, so every single command is already atomic. The trouble starts when your logic needs several commands - read a value, decide, write something back - because another client can slip in between your steps. There are four tools for that, in rising order of power: a single atomic command such as INCR or SET ... NX, a MULTI/EXEC transaction, optimistic locking with WATCH, and a Lua script run with EVAL. Use the simplest one that closes the gap. For most "check, then change" logic that is a short Lua script; for a plain batch of writes that must not be interleaved it is MULTI/EXEC; and WATCH is what you reach for when the decision has to be made in your application code.

This post explains exactly what each one guarantees, which guarantees people assume that do not exist (there is no rollback), and the handful of atomic patterns - counters, compare-and-delete, locks, stock decrement, idempotency keys - that cover most real use. It assumes you can already connect; if not, start with Valkey getting started.

Why single commands are safe and sequences are not#

Valkey executes commands on one main thread. While a command runs, no other client's command runs, which is why INCR counter can never lose an update the way "read, add one, write" in application code can. Your own code is the problem. Consider two workers decrementing stock:

worker A: GET stock:42      -> 1worker B: GET stock:42      -> 1worker A: SET stock:42 0    (sold the last item)worker B: SET stock:42 0    (sold the same item again)

Each command was atomic. The sequence was not. Valkey did exactly what it was told, in an order your code did not control. The fix is never "be faster"; it is to make the read, the decision and the write happen as one unit, or to detect that the data changed underneath you and try again.

Before reaching for anything heavier, check whether one command already does the job. Many do:

SET key value NX EX 30 matters in particular: the older two-step form, SETNX followed by EXPIRE, can leave a key with no expiry if the client dies between the two. The single command cannot. Prefer one command wherever one exists; the rest of this post is for the cases where none does. The data types behind these commands are covered in Valkey data types explained.

MULTI and EXEC: queued, not isolated by logic#

MULTI starts a transaction. Every command after it is not run but queued, and the server answers QUEUED. EXEC then runs the whole queue in order, as one uninterrupted block: no other client's command is executed between the first queued command and the last. You can try it in valkey-cli, where the prompt shows (TX) while a transaction is open:

127.0.0.1:6379> MULTIOK127.0.0.1:6379(TX)> DECR stock:42QUEUED127.0.0.1:6379(TX)> LPUSH orders:pending "order-9001"QUEUED127.0.0.1:6379(TX)> EXEC1) (integer) 02) (integer) 1

EXEC returns one reply per queued command, in order. DISCARD throws the queue away instead and leaves nothing changed.

What this gives you: isolation (nobody sees a half-finished state) and an all-in-one-go execution. What it does not give you, and this is the part people get wrong:

• No reads that feed later writes. Inside MULTI you get QUEUED, not a value. You cannot GET a number and use it in the next queued command, because the GET has not run yet. All the decisions have to be made before you open the transaction.
• No rollback. If one command fails at run time, the others still run. A transaction containing INCR on a key that holds text will report an error for that one command and happily apply the rest.
• Two kinds of failure, treated differently. If a command is malformed when you queue it (unknown command, wrong number of arguments), the server remembers the error and EXEC refuses the entire transaction with EXECABORT. If a command is well-formed but fails when it runs (wrong type of value), it fails alone.

127.0.0.1:6379> SET name "alice"OK127.0.0.1:6379> MULTIOK127.0.0.1:6379(TX)> INCR nameQUEUED127.0.0.1:6379(TX)> SET other 1QUEUED127.0.0.1:6379(TX)> EXEC1) (error) ERR value is not an integer or out of range2) OK

other was set even though INCR name failed. If you came from a SQL database, that is the surprise to plan for: Valkey transactions are a batch that cannot be interrupted, not a unit that can be undone. The reason is a design choice - a rollback needs an undo log, and the project prefers a fast, simple server, on the argument that a command failing at run time is a programming error that should be caught in development.

MULTI is not just pipelining

Pipelining sends many commands without waiting for each reply; it saves round trips but other clients' commands may run in between. MULTI/EXEC adds the guarantee that they do not. Most client libraries blur the two, so check which one you are using:

// ioredis: .multi() is a real MULTI/EXEC; .pipeline() is notconst results = await valkey  .multi()  .decr("stock:42")  .lpush("orders:pending", "order-9001")  .exec();// results: [[null, 0], [null, 1]]  (error, value) per command

# redis-py: transaction=True (the default) wraps the pipeline in MULTI/EXECpipe = r.pipeline(transaction=True)pipe.decr("stock:42")pipe.lpush("orders:pending", "order-9001")print(pipe.execute())        # [0, 1]

In redis-py, pipeline(transaction=False) sends the same commands with no MULTI at all. That is faster and perfectly right for a batch of independent writes, and wrong for anything that needs the no-interleaving guarantee.

WATCH: optimistic locking#

MULTI alone cannot do "read the balance, and only if it is enough, subtract". WATCH supplies the missing piece. You tell the server which keys you care about; if any of them is modified by another client before your EXEC, the transaction is not run and EXEC returns a null reply. You then read again and retry. It is called optimistic because nothing is locked: you assume nobody will interfere and handle the case where somebody does.

The sequence is always the same:

1. WATCH the keys you are about to read.
2. Read them normally (outside MULTI) and decide in your own code.
3. MULTI, queue the writes, EXEC.
4. If EXEC returned null, someone changed a watched key. Go back to step 1.

In Python it looks like this, a withdrawal that refuses to go below zero:

import redisdef withdraw(r, account, amount):    key = f"balance:{account}"    with r.pipeline() as pipe:        while True:            try:                pipe.watch(key)                balance = int(pipe.get(key) or 0)   # immediate mode after WATCH                if balance < amount:                    pipe.unwatch()                    return False                pipe.multi()                pipe.decrby(key, amount)                pipe.execute()                      # raises WatchError if changed                return True            except redis.WatchError:                continue                            # someone wrote to it - retry

A few rules that save debugging time:

• WATCH is cleared by EXEC, DISCARD, UNWATCH and by closing the connection. Whether the transaction ran or aborted, the watch is gone, so the retry has to call WATCH again.
• WATCH belongs to a connection. With a connection pool you must use the same connection for WATCH, the reads and EXEC, which is what the pipeline() context above does. Calling WATCH through one pooled connection and MULTI through another is the classic bug: no error, no protection.
• Any write to a watched key counts, even one that sets the same value. A hot key under heavy contention makes the retry loop spin; cap the number of attempts and fail with a clear error instead of looping for ever.
• Watching a key that does not exist yet is allowed and useful: creating it counts as a modification, which is how you detect two clients racing to initialise the same thing.

WATCH is the right tool when the decision needs application logic that Lua cannot reasonably hold - calling another service, parsing JSON with your own libraries. For logic that fits in a few lines of Lua, a script is simpler and does not need a retry loop at all, because nothing can interrupt it.

EVAL: run Lua on the server#

EVAL sends a Lua script to the server, which runs it as one atomic step. No other command runs while it executes, so the read, the decision and the write cannot be interleaved - no WATCH, no retries, one round trip. The shape of the command is:

EVAL script numkeys key [key ...] arg [arg ...]

numkeys says how many of the following words are key names; those appear in the script as the array KEYS, and everything after them as ARGV. Both arrays start at 1, as Lua arrays do. A script that decrements stock only if there is enough:

-- KEYS[1] stock key; ARGV[1] quantity wantedlocal have = tonumber(redis.call("GET", KEYS[1]) or "0")local want = tonumber(ARGV[1])if have < want then  return -1endreturn redis.call("DECRBY", KEYS[1], want)

Run it from the command line with --eval. Note the lone comma that separates keys from arguments, with a space on each side - that is valkey-cli syntax, not Lua:

$ valkey-cli -h db.example.net -p 6380 --askpass --eval take_stock.lua stock:42 , 3(integer) 17

Or inline, where you give numkeys yourself:

$ valkey-cli -h db.example.net -p 6380 --askpass \    EVAL "return redis.call('GET', KEYS[1])" 1 stock:42"20"

Inside a script you run Valkey commands with redis.call(command, arg, ...). Recent Valkey versions also accept server.call as an alias, but redis.call works on Valkey and on older Redis alike, so it is the safer spelling if your scripts may move. Some details that matter in practice:

• Declare every key you touch in `KEYS`. Scripts that build key names inside the Lua code work on a single server but break on a cluster, where the server must know which keys a script uses to route it. Even if you never expect to cluster, declaring keys costs nothing.
• Values come back as Lua types, and go back as replies. A Lua number is converted to an integer reply (the fractional part is dropped), a Lua table to an array, false to a nil reply. If you need a decimal, return it as a string with tostring.
• `tonumber` before arithmetic. Everything in ARGV is a string, and GET replies are strings.
• Errors. redis.call stops the script and returns the error to the client when a command fails. redis.pcall returns the error as a table instead, so the script can handle it. Neither rolls back writes the script has already made - exactly as with MULTI, a script that fails half-way leaves its earlier writes in place. Do your checks first, writes last.
• Pass the time in as an argument rather than relying on the clock inside the script when you can; it makes the script a pure function of its inputs, which is much easier to test.

EVALSHA and the script cache

Sending the full text of a script on every call wastes bandwidth. SCRIPT LOAD stores the script on the server and returns its SHA1 hash, and EVALSHA runs it by hash:

$ valkey-cli -h db.example.net -p 6380 --askpass SCRIPT LOAD "$(cat take_stock.lua)""9a7f1c0e5b2d8a64c3e19f0b7d25a8c4e6f13b02"$ valkey-cli -h db.example.net -p 6380 --askpass \    EVALSHA 9a7f1c0e5b2d8a64c3e19f0b7d25a8c4e6f13b02 1 stock:42 3(integer) 14

The hash above is illustrative; yours will differ. Three facts about the script cache that cause real incidents:

• It lives in memory and is not saved by the persistence mechanisms, so after a restart EVALSHA answers NOSCRIPT. See AOF and RDB persistence for what a restart does keep.
• Client libraries handle this by trying EVALSHA first and falling back to EVAL with the full text on NOSCRIPT. If you call EVALSHA by hand, you must do the same.
• SCRIPT EXISTS sha [sha ...] tells you whether hashes are cached, SCRIPT FLUSH empties the cache, and the cache is per server, not per database.

Redis 7 and Valkey functions

Newer versions add FUNCTION LOAD and FCALL, which store named, versioned libraries of Lua functions on the server, and those functions are saved with the data and survive a restart. They solve the NOSCRIPT problem at the cost of a deployment step: you load the library when you deploy, rather than letting the client load scripts lazily. For most applications EVAL through a client library is still the simpler path; consider functions when many services share the same server-side logic. Check the version of your server (INFO server) before relying on them.

The cost of atomic: a slow script blocks everyone

Because a script holds the single main thread, a slow script stalls every other client. Keep scripts to a handful of commands with bounded work. Never loop over an unbounded set (KEYS *, a huge list, a SMEMBERS on a million members) inside one. The server has a script time limit, five seconds by default (the setting is lua-time-limit, which newer versions also call busy-reply-threshold); past that, it starts answering other clients with BUSY and permits SCRIPT KILL, which only works if the script has not yet written anything. A script that has already written can only be stopped with SHUTDOWN NOSAVE, which discards everything since the last save. Treat that as a fire extinguisher behind glass, and keep scripts short enough that you never need it.

Atomic patterns worth knowing#

A counter with a ceiling

The common limit check - "increment, but not past 100" - is exactly the kind of read-decide-write that a single INCR cannot express:

-- KEYS[1] counter; ARGV[1] maximum; ARGV[2] ttl in secondslocal n = redis.call("INCR", KEYS[1])if n == 1 then  redis.call("EXPIRE", KEYS[1], tonumber(ARGV[2]))endif n > tonumber(ARGV[1]) then  redis.call("DECR", KEYS[1])  return 0endreturn n

This is the building block of a fixed-window limiter. The full set of variants, including sliding windows and a token bucket, is in rate limiting with Valkey.

Compare-and-delete: releasing a lock safely

A lock in Valkey is a key you create with SET lock:report <token> NX PX 30000: it only succeeds if nobody holds the lock, and it expires after 30 seconds so a crashed holder cannot block everyone for ever. The token is a random string unique to the holder. Releasing is where it goes wrong. If you simply DEL the key, you may delete a lock that has since expired and been taken by someone else. You have to delete it only if it still holds your token, and that is a read followed by a write:

-- KEYS[1] lock key; ARGV[1] the holder's tokenif redis.call("GET", KEYS[1]) == ARGV[1] then  return redis.call("DEL", KEYS[1])endreturn 0

The same shape - compare, then act - gives you "extend my lock if it is still mine" (replace DEL with PEXPIRE) and "ack this job if I still own it". This lock protects against two workers racing on one server. It is not a guarantee of mutual exclusion across a network partition or a long pause in the holder, so for work where a duplicate would do real damage, make the work itself idempotent as well.

Check-and-set with a version

Optimistic concurrency on a single record, without WATCH: store a version number inside a hash and only write when it matches what you read.

-- KEYS[1] hash; ARGV[1] expected version; ARGV[2] new valuelocal v = redis.call("HGET", KEYS[1], "version")if v ~= ARGV[1] then  return 0endredis.call("HSET", KEYS[1], "value", ARGV[2])redis.call("HINCRBY", KEYS[1], "version", 1)return 1

Your application reads HGETALL, makes its change, and calls the script with the version it saw. A zero return means someone else got there first, and the application decides whether to re-read and retry.

Idempotency keys

When a request can be retried (a webhook, a payment callback, a queue message delivered twice), record that you have handled it and skip the repeat. One command does it:

$ valkey-cli -h db.example.net -p 6380 --askpass SET idem:evt_1042 1 NX EX 86400OK$ valkey-cli -h db.example.net -p 6380 --askpass SET idem:evt_1042 1 NX EX 86400(nil)

OK means you are first and should do the work; (nil) means it was already claimed. The caveat is failure between claiming and finishing: if the worker dies after the claim, the retry is refused. Store a status ("started", "done") rather than a bare marker if that matters, and give the claim a TTL that is long enough for the work and short enough to retry after a crash. Job queues with Valkey covers how queue libraries handle the same problem.

Moving data between keys

Moving an item from a pending list to a processing list is a single LMOVE, atomic by itself. Moving a member between sets while updating a counter is a case for MULTI/EXEC, since all of the arguments are known before you start:

MULTISMOVE tasks:todo tasks:doing task-77HINCRBY stats:tasks doing 1EXEC

If any of it depended on a value you had to read first, use a script instead.

Choosing between them#

Lua is not always the best answer. A script is code that lives on the server, outside your usual tests and deploys, in a language your team may not write daily. It is harder to debug, it blocks while running, and it is invisible to anyone reading only the application. Use it when atomicity genuinely requires it, keep each script small, store scripts as files in your repository, and name them for what they do. If you find yourself writing a hundred lines of Lua, the logic probably belongs in the application with a different data model, or in a database with real transactions. The honest summary of Redis: when you need it applies here too: the most reliable atomic operation is the one you did not need.

Transactions, replication and persistence#

A transaction or script is written to the append-only file and sent to replicas as the commands it actually executed, so after a restart the log replays the effects rather than running your logic again. That is why a script that used the clock or a random number does not diverge on recovery. Two consequences:

• A transaction is atomic with respect to other clients, not with respect to a crash. If the server dies mid-write, the AOF can end partway through a transaction; on start-up, Valkey will either truncate the tail or refuse to load until you repair it, as described in AOF and RDB persistence. Do not build anything that needs all-or-nothing durability on top of MULTI.
• With the default appendfsync everysec, the last second of writes can be lost on a hard failure, transactions included.

Access control applies as well. A user restricted by ACL rules needs permission for EVAL/EVALSHA (the @scripting category) and for every command and key the script touches. If a script fails with a permission error when the same commands work by hand, that is where to look; see Valkey security and ACL.

Running this on a RE:NODE Valkey server#

Valkey on the database line is the open-source fork of Redis, speaks the Redis protocol, and is password protected with credentials generated per server. It saves to disk with AOF and snapshots, so the effects of your transactions and scripts survive a restart as described above. Connect with valkey-cli or any Redis client using the host, port and password from the panel, and everything in this post is run the same way: the commands above use the db.example.net:6380 placeholder, replace it with your own host and port. Plans start from $2 a month, and the smallest is 256 MB, which is plenty for counters, locks and idempotency keys. If a script is blocking your server, the console graphs for memory and CPU against the plan's limits show it quickly; a CPU line pinned at its limit while latency climbs is the signature of an expensive script.

FAQ#

Does Valkey support rollback?

No. DISCARD abandons a queue before EXEC, and a transaction with a malformed command is refused whole, but once commands run, a failure part-way does not undo the earlier ones. The same is true of a Lua script that errors after writing. Validate inputs first and put the writes last.

Is MULTI/EXEC the same as a pipeline?

No. A pipeline only batches the sending of commands to save round trips, and other clients' commands can run between yours. MULTI/EXEC guarantees the queued commands run together without interleaving. Some libraries call both "pipeline", so check whether your code actually issues MULTI.

When should I use WATCH instead of a Lua script?

When the decision cannot be made inside Lua, because it needs application code or a call to another system between the read and the write. For anything that fits in a short script, Lua is simpler and needs no retry loop. Under heavy contention on one key, a script also wins, since WATCH retries pile up.

Why do I get NOSCRIPT after a restart?

The script cache is held in memory and is not persisted, so EVALSHA cannot find a script that was loaded before the restart. Run SCRIPT LOAD again, or let your client library fall back to EVAL with the full text, which most do automatically. Functions loaded with FUNCTION LOAD are saved with the data instead.

Can a Lua script call another script or use blocking commands?

Scripts cannot call EVAL from inside a script, and blocking commands such as BLPOP do not block in a script; they behave as if the timeout expired. Keep the script to non-blocking commands and do your waiting in the client.

How long may a script run?

Not long. The default limit is five seconds before other clients get BUSY errors, but any script that takes more than a few milliseconds is already slowing everyone else down. Bound the work a script does, and move large batches into several calls instead of one.