Character Encoding: Difference between revisions
| Line 25: | Line 25: | ||
A '''code point''', or a '''code position''', is any of the numeric values that make up the [[#Code_Space|code space]]. | A '''code point''', or a '''code position''', is any of the numeric values that make up the [[#Code_Space|code space]]. | ||
Some character encoding schemes, such as ASCII, have a fixed relationship between any of the represented characters and the sequence of bits used to represent that character. | Some character encoding schemes, such as ASCII, have a fixed relationship between any of the represented characters and the sequence of bits used to represent that character. For example, the ASCII [[#Code_Space|code space]] consists of 2<sup>7</sup> = 128 code points and each of the represented characters correspond to a code point and to a predefined bite sequence. | ||
Unicode breaks the assumption that each character should always directly correspond to a particular sequence of bits. Instead, the characters are first mapped to a universal intermediate numeric representation. The numeric values characters are mapped to are called '''code points'''. Code points could then be represented in a variety of ways and with various numbers of bits per character ([[#Code_Unit|code unit]]), depending on the context. | Unicode breaks the assumption that each character should always directly correspond to a particular sequence of bits. Instead, the characters are first mapped to a universal intermediate numeric representation. The numeric values characters are mapped to are called '''code points'''. Code points could then be represented in a variety of ways and with various numbers of bits per character ([[#Code_Unit|code unit]]), depending on the context. | ||
Revision as of 21:43, 25 June 2018
External
Internal
Overview
Character encoding is the process though which characters within a text document are represented by numeric codes and ultimately translate into sequence of bits stored or sent over the wire. Depending of the character encoding used, the same text will end up with different binary representations. Common character encoding standards are ASCII, Unicode and UCS.
Concepts
Character
Depending on the encoding scheme used, characters may be represented with different code units - the number of bits used to represent a single character.
Character Set
Character Code
Unicode is a character code.
Code Point
A code point, or a code position, is any of the numeric values that make up the code space.
Some character encoding schemes, such as ASCII, have a fixed relationship between any of the represented characters and the sequence of bits used to represent that character. For example, the ASCII code space consists of 27 = 128 code points and each of the represented characters correspond to a code point and to a predefined bite sequence.
Unicode breaks the assumption that each character should always directly correspond to a particular sequence of bits. Instead, the characters are first mapped to a universal intermediate numeric representation. The numeric values characters are mapped to are called code points. Code points could then be represented in a variety of ways and with various numbers of bits per character (code unit), depending on the context.
Combining Character
Code Unit
The number of bits used to represent a character within a certain encoding scheme. For example, ASCII has a code unit of 7, UTF-8 a code unit of 8, etc.
Code Space
Character Encoding Standards
ASCII
ASCII stands for American Standard Code for Information Interchange and it is a seven-bit encoding scheme used to encode letters, numerals, symbols, and device control codes as fixed-length codes using integers. ASCII consists of 27 = 128 code points in the range 0x0 to 0x7F. ASCII is specified by ISO 646 IRV.
Extended ASCII
Extended ASCII consists of 28 = 256 code points in the range 0x0 to 0xFF.
Unicode
Unicode supports a much larger character set than ASCII. It comprises 17 x 65,536 = 1,114,112 code points in the range 0x0 to 0x10FFFF. The Unicode code space is divided into seventeen planes, of which the plane 0 is the Basic Multilingual Plane, and 16 supplementary planes, where each plane supports 216 = 65,536 code points.
Unicode Transformation Format (UTF)
Binary representation of a text represented in Unicode depends on the "transformation format" used. UTF stands for "Unicode Transformation Format", and the number specified after the dash in the transformation format name represents the number of bits used to represent each character.
UTF-8
UTF-16
UTF-16 support a large enough character set to represent both Western and Eastern letters and symbols.