Standard

Rules for Editing This Page

• Only registered users can edit. You can register here.
• If you have a change you want to perform, simply perform it.
  • If you suspect the change requires explanation, place it in the discussion page.
• This page is for the actual standard. Discussions about the standard, open questions, etc., go in the discussion page.

Introduction

This document intends to trace the guidelines of a User Interface (in short: UI) for editing bidirectional (in short: Bidi) text. It is assumed that the user enters text in logical sequence, and that the Unicode Bidi Algorithm (in short: UBA) is used to reorder the text for presentation. We assume that the readers of this document have a working knowledge of the UBA. The UBA is described in Unicode Technical Report 9 (see http://www.unicode.org/unicode/reports/tr9).

When designing these guidelines, the following objectives were set, in order of decreasing priority:

1. Prevent actions unexpected by the user, particularly when the action is destructive (erases one or more characters).
2. Make the interface efficient.
3. Keep the interface easy to implement.

Scope

This standard deals with BiDi aspects of text editing. Every attempt has been made to make sure that it does not impose one editor model on implementors. Different implementations differ in parameters, such as whether it is allowed for the caret to reside outside the buffer space, behavior when moving past the buffer, and many others. It is not the intent of this standard to impose homogeneous behavior, but rather to resolve the BiDi aspects of text editing. Wherever possible, such differences are simply not mentioned in this standard.

Sometimes, these differences carry BiDi implications. For example, some implementations move the caret to the beginning of the line when an up arrow is pressed while the caret is on the first line of the text buffer. This standard does define what is the correct way to perform this caret move. This is not to say that the standard mandates, or even recommends, that the caret move in such a way. Only that, should it move, it should move in a certain way.

Notational Conventions

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

Text Editing vs. Word Processing

There are two broad categories for editing bidirectional text. One type is a text editor, and another is a word processor. The main difference, for our purposes, between the two is that a text editor produces pure text output, to be displayed strictly according to the Unicode Bidi algorithm, while a word processor is allowed to format the text in more liberal fashion. Also, a text editor is expected to output only and exactly what the user typed, while a word processor is allowed to shape the text using the UBA's control characters, or any other way.

It should be noted that, while the standard does not mandate that word processors use the UBA to format text, use of the UBA is assumed in the text of this standard. Implementors SHOULD use the UBA to order the text, but if they do not, implementors MUST carry out the standard so that the end result is as if implementors did implement the UBA. For example, when we state that a word processor, when copying text from an RTL paragraph into an LTR paragraph, must embed that text within an RLE/PDF sequence, the word processor MAY use any other mechanism other than an RLE/PDF pair, but still MUST make sure that the pasted text is treated as a RTL embedded sequence. Also, when exporting pure text representation of the output, placing an RLE/PDF pair around the text is the only way to ensure it will be displayed correctly by text editors and other output means.

Terms and Definitions

Abbreviations

Acronym	Meaning
Bidi	bidirectional
LTR	left-to-right
RTL	right-to-left
UBA	Unicode Bidi Algorithm
UI	User Interface

Definitions

Bidi Embedding Levels	The UBA assigns a level to each character in the logical buffer, including neutrals, which determines if it is part of LTR or RTL text, and eventually affects the presentation. Level 0 corresponds to base LTR text. Level 1 corresponds to base RTL text, or to RTL text embedded within level 0 LTR text. Level 2 corresponds to LTR text embedded within level 1 RTL text, itself possibly embedded within level 0 LTR text. And so on for higher levels. Even levels always correspond to LTR text, odd levels always correspond to RTL text.
Caret (aka Text Cursor)	Graphic representation of where actions like text entry or Delete are going to take effect. The caret is often displayed as a vertical bar. The text cursor is not to be confused with the mouse cursor. Throughout this document, the term "caret" refers to the text cursor, and the term "cursor" refers to the mouse cursor.
Cursor Level	For the needs of the UI, a Bidi level is assigned also to the cursor. This level reflects the Bidi level which is expected to be assigned to the next character entered (there are cases when the actual level of the entered character will be different). The level of the cursor is manipulated by UI functions, like changing the keyboard language. It may also be affected by all functions which change the position of the cursor.
Keyboard Language	Language of the next character that will be entered from the keyboard.
Logical Buffer	Buffer containing the text data in logical sequence (as opposed to visual sequence).
Paragraph Embedding Level	Bidi level of text belonging to the main language used in a paragraph. This is 0 if the main language is LTR, 1 if the main language is RTL. Note: there is a one-to-one correspondence between the paragraph embedding level and the "Base Direction", which is the direction of the main language of a paragraph.

Caret

The caret's role is to represent, to the user, where the next action is going to take place. Common implementations use two types of carets. One type, which in modern UIs is usually referred to as "insert mode", is represented as a vertical line drawn between characters. We shall refer to this form as a "line caret". The other, typically representative of overwrite mode, is displayed as an underline beneath the character, or as a block which highlights the character. We shall refer to it as "block caret".

Some archaic implementations use a block caret for both insert and overwrite mode. Modern implementations, however, use the line caret almost exclusively. This document assumes a line caret mode unless explicitly stated otherwise.

A line caret occupies zero space, and is always between two displayed characters (glyphs). We call this position the "caret location". Though the caret location is, logically, of zero width, display does need some actual width in order for the caret to display. See open issues for discussion.

Logical vs. Visual Operations

Throughout this document, there are references to a "visual caret" and a "logical caret". The distinction between the two is an important one. As written above, the caret is a visual indication of where actions take place. For Bidi text, it often happens that a single position in the logical buffer can be interpreted to refer to two (or more) visual positions. Typically, this is a result of whether the caret should be interpreted to be before the next character, or after the previous one. Likewise, a single visual position might, under some circumstances, be interpreted to refer to two positions in the logical buffer.

At the time of this writing, all editors implement a caret that acts as a visual aid to indicate the position in the logical buffer. The caret has a definite and well-defined position in the logical buffer, and its visual position is affected from a variety of considerations. We call such a caret a "logical caret".

This document also defines a new type of caret, called a "visual caret". A visual caret has a definite and well defined visual position. That visual position may, under some circumstances, translate to more than one positions in the logical buffer. A visual caret shall, sometimes, also have a logical position.

Throughout this document, almost any operation will affect the caret. After each operation the caret will either be a visual caret or a logical caret. In other words, the operation will either leave a well defined logical caret position or a visual caret position. An operation that sets the logical caret position is called a "logical operation", and the caret after the operation is a logical caret. An operation that sets the visual caret position is called a "visual operation", and the caret after it is a visual caret. Certain operations are visual operations, which result in a visual caret, but also set the logical position.

References like "before", "after", "first" and "last" rely on the order of characters in the logical buffer, and thus imply logical placement. For example, the sentence "Pressing the END key MUST bring the caret to the last character of the line" means that the caret following pressing "END" is a logical caret. Likewise, references such as "left", "right", "up" and "down" rely on the visual arrangement of the characters on screen. The sentence "Pressing the LEFT key SHOULD move the caret one character to the left" means that the caret following pressing "LEFT" is a visual caret.

A position set by a logical operation is called a "logical position". Likewise, a position set by a visual operation is called a "visual position". A logical position that has only one visual representation can also be said to be a visual position that has only one logical representation. Such a position is agnostic to whether the operation that set it was a visual operation or a logical operation. We call such an agnostic position an "unambiguous position".

Ambiguous Caret

An ambiguous caret is a caret, either logical or visual, that is not located on an unambiguous position. The practical upshot of this is that the implementation needs to choose where on screen to draw the logical caret, or when in the buffer to perform the operation for a visual caret. This section deals with how to determine the different points, and explains how to resolve the ambiguity.

An ambiguous caret will happen if and only if the character before/to the left of the caret has a different BiDi level than the character after/to the right of the caret. If the caret is before the first or after the last character of the paragraph, the missing character's BiDi level is sor or eor respectively, where sor and eor are defined by the UBA. For all practical purposes, sor and eor are at either level 0 or 1, depending on the paragraph base direction (0 for left to right, 1 for right to left). The difference between the BiDi levels on both sides of the caret is called the level differential.

Ambiguous Logical Caret

An ambiguous logical caret will have two visual positions. One position is right after the character before the caret, and the other is right before the character after the caret. In this sentence, "after" and "before" do not bear their usual meaning, as visual positions are involved. Instead, "after a character" means to the right of the character if that character's BiDi level is LTR (even), or to the left of the character if that character's BiDi level is RTL (odd).

Examples:

english |ARABIC more english.
00000000 11111100000000000000

Where the caret is represented by the horizontal line. The caret might, visually, be in the following positions:

english |CIBARA| more english.

Another example. In an RTL paragraph, the following logical text:

|english HEBREW more english.
 2222222111111112222222222221

The caret may be in the following positions:

.more english WERBEH |english|

Choosing Correct Visual Position

The visual location a specific logical position translates MUST be one of the locations determined above. Selecting which one should be done according to the following criteria in descending order of precedence:

1. If the last operation changed the buffer, we choose the visual position such that, if the next operation will be the same operation, the caret will be displayed where that operation takes place.
   • If the last operation was to type something, position the caret where the next character will appear assuming it is of the same BiDi class as the previous one.
   • If the last operation was to delete a character with backspace, position the caret so that it is next to the character to be deleted next if a backspace is hit again.
2. If the last operation did a logical movement with no change to the buffer (HOME, END etc.), position the caret next to the character the caret skipped over.
   • If the move was to an earlier (logical) position, position the caret next to the character immediately after the caret.
   • If the move was to a later position, position the caret next next to the character immediately before the caret.

See open issues for discussion

Caret Movement

This section covers operations explicitly designed to move the caret around. All operations under this section do not change the logical buffer.

Positioning Using a Pointing Device

The caret can be positioned using a pointing device. This is, typically, done by clicking with a mouse in or near the text area. Since a pointing device usually has a pixel accuracy, some rounding will be performed. Implementations SHOULD select the caret location that is nearest to the point where the pointing device asked to locate the caret. Implementations SHOULD NOT care whether the rounding is to the left or to the right.

This is a visual operation.

Left and Right Arrow Keys

Pressing the left key MUST move the caret one character to the left. Pressing the right key MUST move the caret one caret to the right.

When a left arrow is pressed while the caret is already at the left most character of the line, the caret SHOULD move to the right most side of the line above or below, depending on the paragraph direction. If the paragraph is an LTR paragraph, pressing the left arrow key while the caret is on the left most character of the line SHOULD move the caret to the right most character of the line above. Likewise, if the paragraph is an RTL paragraph, pressing the left arrow while the caret is on the left most character of the line SHOULD move the caret to the right most character of the line below the current line.

The situation for right arrow on right most character is the reverse of the above. On LTR paragraphs, pressing the right arrow while the caret is on the right most character of the line SHOULD move the caret to the left most character of the line below the current line. On RTL paragraphs, pressing the right arrow while the caret is on the right most character of the line SHOULD move the caret to the left most character of the line above the current line. See open issues for discussion of alternatives

This is a visual operation

Home and End Keys

On most platforms, the "Home" key places the caret at the beginning of the line it is currently on, and "End" places it at the end of the same line. The end of the line means positioning the caret right after the last character in the logical buffer that is displayed on the current line. Likewise, the beginning of the line means positioning the caret right before the first character in the logical buffer that is displayed on the current line.

On some platforms, most notably Apple Macintosh, the same functionality is achieved using a modifier over the arrow keys (command-right and command-left). On such platforms, the meaning is the same as above, except the mapping between command-right/left to home/end depends on the paragraph direction. If the paragraph is left to right, command-left translates to the behavior discussed above under "Home", and command-right to "End". If the paragraph is right to left, command-left translates to "End" and command-right to "Home". See open issues

In either cases, this is a logical operation.

Start/End of Paragraph/Document

Some implementations have a key combination for moving the caret to the beginning/end of the current paragraph, or the beginning/end of the entire document.

Moving to the beginning of the paragraph means positioning the caret right before the first character of the paragraph. Likewise, moving to the beginning of the document means positioning the caret right before the first character of the document.

In a similar way, moving to the end of the paragraph/document means positioning the caret right after the last character of the paragraph/document.

This is a logical operation.

Up and Down Arrow Keys

Pressing the "up" or "down" keys MUST move the caret one line up or down, respectively. The horizontal caret location is determined by the same algorithm as in "Positioning Using a Pointing Device" above.

If the caret is already on the first/last line of the text buffer, some implementations choose to move it to the beginning/end of the text buffer. If this is the case, implementations MUST use the same as logic as in the "Start/End of Paragraph/Document" section above. If that is the case, the operation MUST NOT be visual.

In the normal case, this is a visual operation.

Page Up and Down Keys

The page up and page down operations SHOULD behave exactly like repeated up and down operations respectively.

See the previous section to determine whether this is a visual or logical operation.

Linear Selection

Selection is a process of highlighting part of the text. Selection is not, in itself, an operation. Rather, selection is a prerequisite step for other operations, such as deleting a section of text, or copying text into the clipboard.

This standard only deals with the most basic form of selection there is, which is the linear selection. A linear selection is defined by a start and an end position, selecting all text in between the start and end position. Certain editors also support other forms of selections (most notable are the multi-linear and the rectangular selections). These are not covered by this standard.

Linear Selection is a contiguous subsection of the logical buffer. The selection area MUST be continuous in the logical buffer. Of course, due to mixed directional runs, this means that the selection's highlight might be non-continuous. Editors SHOULD NOT constraint the number of visually continuous sequences a selection highlight might take, as real life is almost guaranteed to surprise you. If an upper bound must be used, then the following formula should be used:

• The first and last line of the selection, each, might contain as many visual highlight blocks as there are BiDi levels. The UBA limits the number of allowed BiDi levels to 63 (zero through 61, plus automatic increase defined by the "I" rules defined in section 3.3.5), and if a maximum must be defined, this is the number that SHOULD be used as the maximum for each line (first and last).
• If the selection only spans one line, that single line might have as many as twice the number of highlight blocks less one (i.e. - a maximum of 125 highlight blocks).
• If the selection spans more than two lines, all lines except the first and last will be completely and continuously highlighted.
• Implementation SHOULD assume that the entire 64 levels might be used. In any case, implementation MUST NOT assume that less than three BiDi levels are used, as that is the number of BiDi levels that can be reached with no BiDi control characters at all. This allows for up to five highlight blocks in a single line.

The following image shows text with no BiDi control characters, three BiDi levels and five highlight sequences. The selection starts at the left numbers block, between the "12" and the "34", and ends at the right numbers block, between the "12" and the "34":

The text without selection is "English עברית 1234 ומספרים more English עוד עברית 1234 מספרים and that's it."

Selection Beginning and End Points

While the selection itself is a logical operation, selecting a logically continuous region of text, the determination of the beginning and end points for the selection may or may not be logical, depending on the way in which they are selected. The selection is usually performed either by a mouse drag (i.e. - mouse press on the start point, drag the mouse to the end point, and depress the button) or a special movement modifier (such a shift + arrow key). The selection beginning and end positions are, individually, visual if the operation that set them there was visual and logical if the operation that set them there was logical. Again, regardless of how the beginning and end points are chosen, whether visual or logical, the selection itself is continuous in the logical buffer.

Logical Beginning and End Points

If both beginning and end points are chosen using a logical operation (such as Home/End, or as unambiguous positions), then everything is uniquely defined. The selection covers the entire range between the beginning point and the end point.

Logical Beginning and Visual End Points (Or Vice Versa)

For the sake of this discussion, we'll assume that the start point is a logical position, and the end point is visual position which is not an unambiguous position. The exact same considerations apply in the opposite case.

The end marker MUST be set to be on the logical position that satisfies the following two conditions:

• The last character in the selection range is adjacent to the visual position of the marker.
• The marker does not have highlighted characters on both its sides.

All visual positions have only one logical position that satisfies both these requirements. That position is the logical position to use for the end position.

Index of Operations

This section lists all of the operations defined in the document above. Each operation lists the section in which it is defined, as well as whether it is a logical or visual operation (i.e. - whether the caret following performing it is a visual or logical caret).

L/V	Operation	Section	Description
L	Filler	name	Just a filler operation, until we have an actual operation to place here