URL Encoding: How Percent-Encoding Works

A URL may contain only a small set of ASCII characters, and several of them have a specific job: / separates path segments, ? starts the query, & separates parameters and # marks the fragment. As soon as a piece of data contains a space, an accented letter or one of those characters, it has to be encoded. URL encoding, also called percent-encoding, is defined in RFC 3986. What it really says to the server is: "this character is part of the value, not a separator".

The principle: one byte becomes three characters

An encoded character is written as % followed by two hexadecimal digits that give the value of one byte (RFC 3986, section 2.1). The space has the code 32, which is 20 in hexadecimal, so it becomes %20. Encoding any piece of text takes three steps:

1. Convert the text to UTF-8 bytes.
2. Leave the bytes that stand for unreserved characters as they are (ASCII letters, digits and - . _ ~).
3. Replace every other byte with % and its hexadecimal value, preferably in uppercase.

A character outside ASCII takes up several bytes in UTF-8, which is why a single letter can turn into several percent groups:

So "café & thé" becomes caf%C3%A9%20%26%20th%C3%A9. UTF-8 is not an arbitrary choice: RFC 3986 recommends it (section 2.5), and it is what browsers and modern languages use. Some older systems encoded text as Latin-1, where é is a single byte and becomes %E9. A UTF-8 decoder rejects that sequence: in JavaScript, decodeURIComponent('%E9') throws URIError: URI malformed.

Reserved and unreserved characters

RFC 3986 sorts characters into two groups:

• Unreserved (section 2.3): A-Z a-z 0-9 - . _ ~. They never need encoding, and the standard asks you not to encode them: %41 and A mean the same thing.
• Reserved (section 2.2): the general delimiters : / ? # [ ] @ and the sub-delimiters ! $ & ' ( ) * + , ; =. They give the URL its structure.

The rule fits in one sentence: a reserved character is written as-is when it acts as a separator, and encoded when it belongs to the data. Compare:

https://example.com/search?q=bread&milk      -> q = "bread", plus an empty parameter "milk"
https://example.com/search?q=bread%26milk    -> q = "bread&milk"

Every other character, including spaces, quotation marks, % itself, accented letters and emoji, must be encoded wherever it appears. The % sign is therefore written as %25: "100 %" with a space becomes 100%20%25.

The space: %20 or +?

Under RFC 3986, a space is encoded as %20. HTML forms, however, use a different format, application/x-www-form-urlencoded, in which a space becomes + and a real plus sign has to be written as %2B. Browsers use it for forms sent with GET, and most servers decode query strings this way.

That leads to a common mix-up. In a path, + is just a plus sign. In a query string it is usually read as a space: a search for "C++" sent without encoding, ?q=C++, reaches the server as "C" followed by two spaces. Encoded correctly, it is ?q=C%2B%2B.

Both forms are correct in a query string, as long as the side that decodes follows the same convention as the side that encodes.

Encoding in JavaScript: which function to use

JavaScript has two built-in functions that differ in the characters they leave untouched. For a single value, whether a query parameter or a path segment, use encodeURIComponent: it encodes everything except A-Z a-z 0-9 - _ . ! ~ * ' ( ). encodeURI assumes it is given a complete URL whose structure is already correct, so it leaves separators such as &, / and ? alone, even when they belong to your data. The exact list of characters each function skips, and the cases where encodeURI is still the right tool, are covered in encodeURI vs encodeURIComponent.

When you build a URL from its parts, the safest option is to let the URL API encode each part in its proper place:

const url = new URL('https://example.com/search');
url.searchParams.set('q', 'café & thé');
url.searchParams.set('sort', 'price');
url.href;  // 'https://example.com/search?q=caf%C3%A9+%26+th%C3%A9&sort=price'

Note that searchParams follows the form convention, so the spaces come out as +.

Examples in JavaScript, PHP and Python

Here is the same value encoded and decoded in all three languages, with both conventions for the space:

// JavaScript
encodeURIComponent('café & thé');                 // 'caf%C3%A9%20%26%20th%C3%A9'
new URLSearchParams({ q: 'café & thé' }).toString(); // 'q=caf%C3%A9+%26+th%C3%A9'
decodeURIComponent('caf%C3%A9%20%26%20th%C3%A9'); // 'café & thé'

// PHP (file saved as UTF-8)
rawurlencode('café & thé');                // caf%C3%A9%20%26%20th%C3%A9
urlencode('café & thé');                   // caf%C3%A9+%26+th%C3%A9
http_build_query(['q' => 'café & thé']);   // q=caf%C3%A9+%26+th%C3%A9
urldecode('caf%C3%A9+%26+th%C3%A9');       // café & thé

# Python
from urllib.parse import quote, quote_plus, urlencode, unquote_plus
quote('café & thé')                    # 'caf%C3%A9%20%26%20th%C3%A9'
quote_plus('café & thé')               # 'caf%C3%A9+%26+th%C3%A9'
urlencode({'q': 'café & thé'})         # 'q=caf%C3%A9+%26+th%C3%A9'
unquote_plus('caf%C3%A9+%26+th%C3%A9') # 'café & thé'

Two details are worth knowing. In Python, quote() leaves the slash alone by default: quote('a/b') returns a/b. To encode a standalone value, pass safe='', which gives a%2Fb. In PHP, urlencode() also encodes the tilde as %7E, whereas rawurlencode() leaves it as it is, as RFC 3986 requires.

Double encoding

Encoding a string that is already encoded turns every % into %25. %20 becomes %2520, and caf%C3%A9 becomes caf%25C3%25A9. With the form convention, it is the + that gives the problem away: in PHP, urlencode(urlencode('a b')) returns a%2Bb, which the server decodes to "a+b" instead of "a b".

The cause is almost always the same: two layers of code encode the same value, for example an HTTP library that encodes parameters you have already encoded yourself. The same trap exists on the decoding side: in PHP, $_GET already holds decoded values, and calling urldecode() on them decodes a second time, which among other things turns legitimate + signs into spaces. The rule: encode the raw value once, at the moment it goes into the URL, and decode it once when you read it.

Only one case justifies something that looks like double encoding: a URL passed as a parameter of another URL, for example in redirect=. If the inner URL already contains %20, its encoded form contains %2520, and that is correct: the receiver decodes once and gets the original URL back.

Encode and decode online

To check a value without writing any code, the URL Encoder offers both modes, encodeURIComponent and encodeURI, so you can compare their results on your own text. The URL Decoder decodes as you type and reads + as a space, the way forms do, so a real plus sign has to be entered there as %2B. Everything happens in your browser.