Keystores and Mnemonics: Storing the Secret

Every page in this part has orbited a single 32-byte number. secp256k1 keys showed that the private key is just a random integer, and the public key a point derived from it. Deriving an address turned that public key into 20 bytes anyone can pay to. ECDSA signatures and recovery showed that whoever holds the private key can authorize any transaction from that address, and no one else can. The whole edifice rests on one secret staying secret.

Which leaves the least glamorous and most consequential question in the book: where does that number actually live? A human cannot memorize 32 random bytes, cannot type them without error, and must not paste them into a text file where malware can read them. This page is about the two mechanisms wallets use to bridge the gap between a raw cryptographic secret and a fallible human who has to store it, back it up, and — years later — recover it. Both are pure client-side conventions: the chain never sees them. But get them wrong and the throughline of this book turns against you. In a network of untrusting strangers there is no support desk, no password reset, and no one who can restore your account. The secret is the account.

Two different jobs: protect-at-rest and back-up

It helps to separate two problems that wallets solve with two different tools, because people conflate them constantly.

   raw private key (32 bytes)
        │
        ├──▶  KEYSTORE  ── encrypt under a password ──▶  safe to store on disk
        │                  (protect the key on a device you use every day)
        │
        └──▶  MNEMONIC  ── encode the entropy as words ──▶  safe to write on paper
                           (back up the key so you can recover if the device dies)

A keystore answers “how do I keep this key on my laptop without leaving the raw bytes lying around?” It encrypts the key under a password. The file is useless without the password.
A mnemonic answers “how do I back this up so I can restore it on a new device — or hand it to my heirs — without copying 32 hostile hex characters by hand?” It turns the underlying randomness into an ordered list of ordinary words.

They are not alternatives; a real wallet often uses both. But they defend against different failures — a stolen laptop versus a dead one — so we take them one at a time.

Keystores: the key, encrypted under a password

A keystore is the oldest answer, and the simplest to state: take the 32-byte private key, encrypt it with a symmetric cipher under a key derived from your password, and write the ciphertext (plus everything needed to decrypt it except the password) to a JSON file. Geth calls this the “Web3 Secret Storage” format, and it is the .json file every classic Ethereum wallet drops in its keystore directory.

The naive version — “encrypt the key with AES using the password as the AES key” — is broken, and understanding why is the whole lesson. Passwords are low-entropy. A human password might carry 30–40 bits of real randomness; a 256-bit AES key wants 256. If you used the password directly, an attacker who steals the file could simply try every candidate password, and passwords are guessable fast — billions per second on a GPU. The ciphertext would fall in seconds.

So the keystore inserts a deliberate speed bump: a Key Derivation Function (KDF).

The KDF: making each guess expensive on purpose

A KDF takes your password plus a random salt and grinds them, slowly and by design, into the actual encryption key. “Slowly” is the feature. The KDF is tuned so that deriving the key from one password guess takes a meaningful fraction of a second and, crucially, a large amount of memory. Two KDFs appear in Ethereum keystores:

scrypt — the default. It is memory-hard: computing it requires filling a large block of RAM, which is cheap for you (one guess) but ruinous for an attacker trying to run millions of guesses in parallel on specialized hardware. GPUs and ASICs are fast at arithmetic but have limited fast memory, so scrypt’s memory cost blunts exactly the hardware attackers reach for.
PBKDF2 — the older, simpler alternative. It just repeats a hash function a configurable number of times (the c iteration count). It is not memory-hard, so it is weaker against GPU attack for the same wall-clock cost, but it is widely supported.

The KDF parameters live in the keystore file, in the clear — the salt, the cost factors, everything an honest wallet needs to redo the derivation. That is safe: knowing the parameters does not help you skip the work. What the file never contains is the password itself.

   keystore.json  (simplified, scrypt variant)
   {
     "crypto": {
       "cipher": "aes-128-ctr",
       "ciphertext": "…the encrypted private key…",
       "cipherparams": { "iv": "…" },
       "kdf": "scrypt",
       "kdfparams": {
         "n": 262144,        ← CPU/memory cost  (2^18)
         "r": 8, "p": 1,     ← block size, parallelism
         "salt": "…",        ← unique random salt, defeats rainbow tables
         "dklen": 32
       },
       "mac": "…"            ← integrity check on the derived key + ciphertext
     },
     "version": 3
   }

How decryption (and forgery-detection) actually works

Reading the key back is the reverse of writing it, with one extra guard:

   password ─┐
             ├─▶ scrypt(password, salt, n, r, p) ─▶ derived key (32 bytes)
   salt ─────┘                                         │
                                    ┌──────────────────┴───────────────┐
                                    │                                  │
                     first 16 bytes → AES key            last 16 bytes → MAC key
                                    │                                  │
                    AES-CTR-decrypt(ciphertext) → private key    keccak256(mackey ‖ ciphertext)
                                                                       │  == stored "mac"?
                                                                       ▼
                                                        yes → key is authentic; no → WRONG PASSWORD

The MAC (message authentication code) is what lets the wallet tell you “wrong password” instead of silently handing back garbage. Without it, a wrong password would still produce some 32-byte output from AES — a bogus key that owns nothing. The MAC is keccak256 of part of the derived key concatenated with the ciphertext; if your password was wrong, the derived key is wrong, so the MAC won’t match, and the wallet rejects it. (Notice keccak256 again — the same hash that builds addresses does integrity duty here too.)

   scrypt default:  n = 262144 (2^18), r = 8, p = 1

   memory per guess ≈ 128 · r · n bytes
                    = 128 · 8 · 262144
                    ≈ 256 MiB of RAM per single password attempt

   For you:   one derivation, ~0.1–1 s, 256 MiB — unnoticeable, once.
   For an attacker trying to brute-force:
     to run 10,000 guesses in parallel he needs ~2.5 TiB of fast RAM,
     not just fast arithmetic — and RAM is the resource GPUs/ASICs
     are stingy with. The memory-hardness is what turns "billions of
     guesses/second" into "a few thousand, if you can afford the RAM."

A weak password still loses eventually — the KDF buys time proportional to guess cost, not immunity. Against a strong, high-entropy password the slowdown makes brute force astronomically expensive; against password123 it merely delays the inevitable by a few minutes.

The security model is now sharp. A stolen keystore file is not a stolen key — it is a stolen puzzle whose difficulty is set by your password and the KDF cost. That is genuinely useful for a device you use daily. But it has a fatal ergonomic flaw as a backup: the file is a blob of hex you cannot read aloud, cannot write on paper without transcription errors, and cannot reconstruct if the disk dies. For backup, we need something a human can actually handle. That is the mnemonic.

BIP-39 mnemonics: the wallet, as words

A mnemonic phrase is an ordered list of ordinary English words — 12 or 24 of them — that is your wallet’s secret in a form humans can copy. It comes from BIP-39, a standard Ethereum inherited wholesale from Bitcoin (one of several primitives both chains share). When a wallet says “write down these 12 words and never show anyone,” those words are not a hint or a password to the key — they encode the entropy the key is built from. Anyone holding the words controls every account the wallet derives. There is nothing else to steal.

Here is how random bytes become memorable words.

Step 1 — Start with entropy

The wallet generates raw randomness: 128 bits for a 12-word phrase, 256 bits for a 24-word phrase. This entropy is the real secret — the whole security of the wallet is the security of these bits. (Recall from secp256k1 keys: a private key is just random bits, and “generate a key” means “sample good randomness.” A weak random source here dooms everything downstream.)

Step 2 — Add a checksum

Words a human copies get miscopied. BIP-39 defends against that with a checksum: hash the entropy with SHA-256 and append the first entropy_bits / 32 bits of that hash to the entropy.

   128-bit entropy  ⟶  SHA-256  ⟶  take first 128/32 = 4 checksum bits
   append:  128 entropy bits + 4 checksum bits = 132 bits total

Those extra bits are why a random list of dictionary words is not a valid mnemonic: the last word encodes a checksum over all the others. Mistype one word and, with overwhelming probability, the checksum fails and the wallet refuses to load it — catching the error instead of silently deriving the wrong wallet.

Step 3 — Map bits to words

Split the 132 bits into 11-bit chunks. Eleven bits index a number from 0 to 2047 — and the BIP-39 wordlist is exactly 2048 carefully chosen words (no two share their first four letters, so even a truncated word is unambiguous). Each 11-bit chunk selects one word.

   132 bits ÷ 11 bits per word = 12 words        (for 128-bit entropy)
   264 bits ÷ 11 bits per word = 24 words        (for 256-bit entropy)

   11 bits ─▶ index 0…2047 ─▶ wordlist[index] ─▶ e.g. "ripple"

So a 12-word phrase carries 128 bits of secret entropy plus 4 bits of checksum; a 24-word phrase carries 256 bits plus 8. The word count is a direct readout of how much randomness backs the wallet.

Step 4 — Stretch the words into a seed

The words are memorable but not yet the material keys are derived from. BIP-39 runs the phrase through PBKDF2 with 2048 iterations of HMAC-SHA512, salted with the string "mnemonic" plus an optional user passphrase, to produce a 512-bit seed.

   PBKDF2( password = mnemonic words,
           salt     = "mnemonic" + optional passphrase,
           iters    = 2048, hash = HMAC-SHA512 )
        ⟶  512-bit seed

That optional passphrase (BIP-39 calls it the “25th word”) is a second secret: it never appears in the words, so a shoulder-surfer who photographs your 24 words still cannot derive your accounts without it. Different passphrase, entirely different wallet — a feature sometimes used for plausible deniability. The cost is symmetric: forget the passphrase and the words alone are worthless.

One seed, many keys: HD wallets

The seed is not one private key — it is the root of a whole tree of them. This is the hierarchical deterministic (HD) wallet idea from BIP-32, with a standard tree layout from BIP-44. From the single 512-bit seed, a wallet can deterministically derive a practically unlimited number of key pairs, each at a labeled position called a derivation path.

   mnemonic (12/24 words)
        │  BIP-39 PBKDF2
        ▼
   512-bit seed
        │  BIP-32 tree derivation
        ▼
   master key
        ├── m/44'/60'/0'/0/0   ─▶  private key ─▶ address  (account #1)
        ├── m/44'/60'/0'/0/1   ─▶  private key ─▶ address  (account #2)
        ├── m/44'/60'/0'/0/2   ─▶  private key ─▶ address  (account #3)
        └── …                                    (as many as you like)

The Ethereum path m/44'/60'/0'/0/0 reads as: BIP-44 layout, coin type 60 (Ethereum’s registered number), account 0, external chain, index 0. Bump the final index and you get the next account — same seed, new address, no new backup. This is why importing your 12 words into a fresh wallet restores all your accounts at once: they were never stored separately. They were always just positions in a tree grown from that one seed. It is the single-secret idea from the part overview taken to its conclusion: not one key behind one address, but one phrase behind an entire hierarchy of them.

Under the hood — where the companion crate draws the line

This book’s teaching implementation, ethmini, does not implement keystores, BIP-39, or HD derivation — and, as ECDSA and recovery noted, it does not implement signatures either. Its accounts are addressed directly (Address::from_low_u64(1) mints a readable 0x00…01 for tests), and its Account record is four fields — nonce, balance, storage, code — with no notion of a key at all:

pub struct Account {
    pub nonce: u64,
    pub balance: u128,
    pub storage: BTreeMap<u64, u64>,
    pub code: Vec<u8>,
}

That omission is deliberate and worth naming, because it marks the exact trust boundary this page lives on. Everything in a keystore or a mnemonic happens before a byte reaches the chain: key generation, encryption at rest, backup, and derivation are all client-side wallet software. The chain — the part ethmini models — only ever sees an address and a signature that recovers to it. Whether the human behind that address kept their key in an encrypted JSON file, wrote 24 words on steel, or memorized them, is invisible to consensus. The seam is the point: the network guarantees “only the key-holder can move these funds”; it guarantees nothing about whether the key-holder can find their own key. That second half is entirely on the wallet and the human — which is exactly why it is where funds are actually lost.

The security model has no undo

Stack up what we have built and the consequence is stark. There is no account on a server, no email on file, no “forgot password” link, no fraud department. The mnemonic is the wallet; the wallet is the funds. That yields two irreversible failure modes, and both are total:

   LOSE the mnemonic (and any keystore/passwords):
        the derived keys can never be recovered → funds frozen forever

   LEAK the mnemonic (photo, cloud backup, phishing site, screen share):
        the thief derives every key you will ever have → funds gone, instantly

This is the price of the throughline. “Untrusting strangers agreeing on a shared state” means no one is trusted — including any party who could otherwise help you recover. The absence of a recovery service is not an oversight the ecosystem forgot to build; it is the same property that stops anyone from seizing your account being turned toward you when you lose the key. Self-custody hands you the sovereignty and the liability in one bundle.

That is also the final answer to “why slow KDFs matter.” Once an encrypted keystore leaks — a synced laptop backup, a cloud folder, a discarded drive — the only thing standing between the thief and the key is the cost of guessing your password. There is no rate limiter to trip, no account to lock, no second factor to prompt. The attacker has the file and infinite tries, offline. The KDF’s deliberate slowness, multiplied by a high-entropy password, is the entire remaining defense.

Because the mnemonic is the whole prize, it is the single most phished secret in the ecosystem. After the July 2020 Ledger customer-database breach exposed names, emails, and postal addresses of hundreds of thousands of hardware-wallet buyers, victims were hit for years with targeted phishing: emails and even physical letters impersonating Ledger and urging them to “validate” or “migrate” their wallet by entering their 24-word recovery phrase on a lookalike site. A legitimate wallet never asks for your phrase — but many people typed it in, and their funds were drained within minutes, because the words alone let the attacker derive every key in the tree. The lesson is the security model made concrete: a mnemonic has no revocation and no recovery, so the phrase leaving your possession once, by any channel, is game over. The strongest KDF on earth protects nothing you type into an attacker’s form.

The architect’s lens

Why do they exist? Because a private key is 32 bytes of hostile randomness and the person who owns it is human — they need to store it on a device without leaving it in the clear (keystore) and back it up without hand-copying hex (mnemonic).
What problem do they solve? They bridge a raw cryptographic secret to human hands: the keystore makes an at-rest key survive a stolen file as long as the password holds; BIP-39 makes an unmemorizable secret writable, verifiable (checksum), and recoverable, while BIP-32/44 collapse a whole tree of accounts into that one backup.
What are the trade-offs? Convenience versus a single point of catastrophic failure. One phrase restoring every account is wonderful for recovery and devastating on leak; a memory-hard KDF costs you a fraction of a second but is the only thing between a leaked keystore and a drained wallet.
When should I avoid them? Never store a mnemonic where it can be copied at a distance — a photo, a cloud note, a password manager sync, a screenshot. And never rely on a weak password to protect a keystore that might leak; the KDF only multiplies whatever entropy the password already has.
What breaks if I remove them? Remove the keystore’s KDF and a leaked file is an instantly stolen key. Remove the mnemonic and there is no human-usable backup — a dead device is permanently lost funds. Remove the checksum and a single mistyped word silently derives the wrong wallet with no warning.

Check your understanding

A keystore JSON file is stolen. Why is that not the same as the private key being stolen, and precisely what determines how hard the attacker’s job now is?
Explain why the keystore derives the AES key from the password through a slow, memory-hard KDF instead of using the password as the encryption key directly. What specific attack does the memory-hardness blunt?
Walk the transformation from raw entropy to a 12-word BIP-39 phrase: how many entropy bits, where does the checksum come from and how many bits, and why is the wordlist exactly 2048 words?
Someone has your 24-word mnemonic. How many of your accounts can they access, and why? Connect your answer to the single-seed HD-wallet structure (BIP-32/44).
There is no password reset and no recovery service on Ethereum. Explain how this is the same property that makes the network censorship-resistant, and what that implies for how you must treat both a mnemonic and a keystore password.

Show answers

The stolen file contains only the ciphertext of the key, not the key itself; decrypting it requires the password, which is not in the file. So it is a stolen puzzle, not a stolen secret. Its difficulty is set by two things: the entropy of your password and the cost of the KDF (scrypt/PBKDF2 parameters). A high-entropy password behind a memory-hard KDF makes brute force astronomically expensive; a weak password falls quickly regardless of the KDF.
A password carries far less entropy (~30–40 bits) than a 256-bit key, so using it directly would let an attacker brute-force the ciphertext at billions of guesses per second on a GPU. The KDF forces each guess to be slow and memory-hungry (scrypt fills ~256 MiB per attempt). Memory-hardness specifically blunts parallel hardware attacks — GPUs and ASICs are fast at arithmetic but poor at supplying large amounts of fast RAM per lane, so requiring hundreds of MiB per guess collapses their parallelism.
Start with 128 bits of entropy. Hash it with SHA-256 and append the first 128/32 = 4 bits of that hash as a checksum, giving 132 bits. Split into 11-bit chunks (132 ÷ 11 = 12 chunks), each indexing 0–2047. The wordlist is exactly 2048 = 2^11 words so that 11 bits maps to precisely one word; the words are also chosen to be unique in their first four letters. The checksum is why a random word list isn’t a valid mnemonic and why a mistyped word is usually caught.
All of them — every account the wallet can ever derive. BIP-39 turns the words into a single 512-bit seed; BIP-32/44 derive a whole tree of keys deterministically from that one seed (e.g. m/44'/60'/0'/0/0, …/1, …/2, …). The accounts were never stored separately; they are positions in one tree. So the phrase is the master secret, and holding it reproduces the entire hierarchy of keys and addresses.
Both come from the same root fact: no trusted party holds authority over your account. Because no one can seize or freeze it, the network is censorship-resistant — but for the same reason, no one can restore it for you if you lose the key or reverse a theft if you leak it. That means a mnemonic and a keystore password must be treated as irreplaceable and unshareable: losing them is permanent loss of funds, and leaking them (once, by any channel) is total, instant compromise, with no revocation and no undo.