URL Encoding Explained

URL encoding — more precisely called percent-encoding — is one of those things every developer uses constantly and few could explain from first principles. Why does a space become %20 in one place and + in another? Why does encodeURIComponent mangle a full URL if you run it on the whole string? Here's the mechanism underneath the answers.

The problem it solves

A URL is structured text — :, /, ?, #, and & all carry syntactic meaning, marking where the scheme ends, where the path splits, where the query starts, where one parameter ends and the next begins. The moment a value you want to put inside a URL contains one of those characters literally, the parser can't tell your data from its own structure. A search query of cats & dogs dropped straight into ?q=cats & dogs doesn't give a parameter named q with that value — it gives two parameters, q=cats and dogs=. Percent-encoding fixes this by escaping the characters that would otherwise be misread, so a value can travel through the URL syntax without ever being confused for it.

How it actually works

Every character that needs escaping is replaced with a % followed by two hex digits — the character's byte value in UTF-8. A space (byte 0x20) becomes %20. A ? (byte 0x3F) becomes %3F. Characters outside the ASCII range take more than one byte in UTF-8, so they expand into more than one %XX group — é is two UTF-8 bytes (0xC3 0xA9), so it becomes %C3%A9, not a single escape. This is also why percent-encoding a string full of accented or non-Latin characters can grow it substantially — every non-ASCII character costs at least two, often three, %XX groups instead of one.

Plain ASCII letters, digits, and a small set of punctuation (- _ . ~) are never touched — RFC 3986 calls these unreserved. Everything else is a candidate for encoding, but which characters actually get encoded depends on where in the URL the value sits.

Reserved characters mean different things in different parts of a URL

This is the part that trips people up: a character can be structural in one part of a URL and just an ordinary character in another. / separates path segments, so a literal / inside a single path segment has to be encoded as %2F — otherwise it silently creates an extra segment that wasn't there. But / is completely unremarkable inside a query parameter value, since the query parser doesn't look for it. The same is true of & and = — critical to escape inside a parameter value (or they'll be read as separating parameters), irrelevant everywhere else. There is no single "safe character set" for a URL — there's one per context, and encoding a value for the wrong context is a real source of bugs.

encodeURIComponent vs. encodeURI

JavaScript has two built-in encoders, and picking the wrong one is the single most common mistake:

• encodeURIComponent — for a single piece of data (a parameter value, a path segment) that will be inserted into a URL. It escapes everything except unreserved characters, including /, &, =, and ? — because from the point of view of a single value, all of those are just data that happens to look like URL syntax.
• encodeURI — for a value that's already a complete URL. It deliberately leaves :, /, ?, #, &, and = alone, since those are doing their actual structural job and encoding them would break the URL rather than protect it.

Run encodeURIComponent on a whole URL by mistake and https://example.com/search?q=cats comes out as https%3A%2F%2Fexample.com%2Fsearch%3Fq%3Dcats — technically reversible, but no longer usable as a URL until decoded again. Run encodeURI on a single value by mistake and any & or = it contains stays literal, silently corrupting whatever query string it gets inserted into. (The even older escape() function does neither correctly — it doesn't handle non-ASCII characters per the UTF-8 rule above, and has been deprecated for this since ES3.)

The query string's one exception: + means space

Strictly by RFC 3986, a space encodes as %20, full stop. But query strings specifically follow an older, separate convention — application/x-www-form-urlencoded, inherited from HTML form submissions — where a literal + also means a space. Both %20 and + decode to the same space character inside a query string, but a + means something completely different — itself — everywhere else in a URL. This is exactly why URLSearchParams (used by Query String Parser and URL Builder under the hood) decodes a + in a query value as a space, and encodes outgoing spaces as + rather than %20 — it's following the form-encoding convention on purpose, not being sloppy about the spec. It also means a literal + you want preserved (in an email address, for instance) has to be encoded as %2B inside a query string, or it'll be read as a space instead.

A worked example

Take the search phrase C++ & Rust? as a single query parameter value:

giving C%2B%2B%20%26%20Rust%3F as a query value, or C%2B%2B+%26+Rust%3F using the form-encoding +-for-space convention. Both decode back to the original phrase.

Mistakes worth avoiding

• Encoding the whole URL with encodeURIComponent. Only encode the individual piece you're inserting — the value, not the URL it's going into.
• Not encoding at all, and concatenating strings by hand. Works right up until a value contains a character with special meaning — an & in a name, a / in an ID — and then breaks in a way that's easy to miss in testing and hard to trace in production.
• Encoding a value more than once. Percent-encoding an already-encoded string turns %20 into %2520. This deserves its own explanation — see Common Encoding Problems in APIs for the full breakdown of how it happens and how to spot it.
• Confusing this with Base64. Percent-encoding escapes text that's already text; Base64 represents binary data as text in the first place. They solve different problems and are easy to reach for interchangeably — see Base64 vs URL Encoding if that distinction isn't clear yet.

Try it yourself

Percent-encode or decode text with URL Encode and URL Decode, both offering component and full-URI modes so you can see the difference from the previous section directly. To work with a whole query string instead of a single value, use Query String Parser; to break down or assemble a full URL, see URL Parser and URL Builder. All run entirely in your browser.