regex : Java Glossary

2008-06-20 by Roedy Green ©1996-2008 Canadian Mind Products
index page for letter ⇒ punctuation 0-9 A B C D E F G H I J K L M N O P Q R S T U V W X Y Z (all)

regex

regular expression: a system of pattern masks to describe strings to search for, sort of a more generalised wildcarding. You can use them to scan text to extract data embedded in boilerplate. You can use them to replace boilerplate patterns.

Introduction	Examples
Other Regex Engines	Negative Regex
Quoting, why you need \\\\	Matching vs Finding vs LookingAt
Recipes for Quoting	Splitting
Regex Variations Table	Tips
Multiple Characters	String
Awkward Characters	Books
Terminology	Learning More
Pattern Flags	Links

Introduction

JDK 1.4 introduces the java.util.regex package. If they don’t work, use Wassup to check out the version of Java you are using. You may be inadvertently using an old one. Perl-like Regex expressions are compiled into a Pattern (parsed into an internal state machine format, not byte code). You don’t use a constructor to create a Pattern; you use the static method Pattern.compile(String). Then you create a Matcher object with Pattern. matcher(String) feeding it the String you wonder if matches the pattern. Finally, you call Matcher. matches to see if the xfString fits the pattern. There are many other things you can do, for example, to find multiple matches in your String.

Regex cannot do tasks like look for balanced ( ) or deal with a simple precedence grammar. For that you need a parser.

Other Regex Engines

Daniel Savarese has written a second Reex package based on Perl regexes. Look at the Apache Jakarta project. IBM Alphaworks has one. Search for regex. Jakarta-ORO (née OROMatcher), lets you add regex ability to your own Java programs. Funduc Search and Replace is a utility for doing global search and replace on files using regular expressions. The Quoter Amanuensis helps you compose regex expressions for Funduc Search and Replace. SlickEdit® is a text editor that has supports several kinds of regular expressions for global search and replace. Forté Agent newsreader has a regular expression scheme for decribing junk filters. However, it is completely unlike Java regex. It is more like Google search expressions.

Quoting

Reserved characters, aka meta characters are command characters that have special meaning in regexs must be quoted when you mean them literally, as just characters. This does not mean you must enclose them in quotation marks, but rather you must specially mark them as meant literally by preceding them with a \. e.g. \- \+ \?. If you are unsure, quote. It won’t hurt to quote punctuation that does not need it. However, Don’t quote : in Vslick since \:… has special meaning.

Unfortunately, the regex people used the same quoting character \ as the designers of Java did for String literals. In a non-regex Java String literal, every literal \ must be doubled. In a regex every literal \ must be doubled. So when you express a regex as a Java String literal, every literal \ must be quadrupled! and written as \\\\.

When you compose a regex String on the fly, character by character, then Java String literal quoting iis no longer at play. There you merely need double each \. Be especially careful with File.fileSeparatorChar in composed on the fly regexes. If it is \ it must be doubled.

Java 1.4.1+ also offers \Q… \E quoting long passages without having to quote command characters individually. You still have to quote for String literals though.

The quoter amanuensis will let you compose your literal regex strings then convert them to deal with both regex and Java \\ quoting.

In JDK 1.5+, Pattern.quote will do the same thing the quoter amanuensis does to a String to give you the equivalent regex, properly quoted to match it literally. It just mindlessly sandwiches the string in \Q … \E, whether it needs it or not.

Again, it won’t hurt to quote punctuation that doesn’t need it. Note that " and ' don’t need regex quoting, though they need Java quoting.

Recipes for Quoting Awkward Characters in Java Regexes in Java source code String Literals

How to Write Awkward Characters Literally in Java Regex String Literals
Character name	Character	Java literal	Regex	Java literal + Regex
left bracket, acting as a regex command character	[	"["	[	"["
left bracket, reserved regex command character acting as a literal [	[	"["	\[	"\\["
A literal newline character	␊	"\n"	\n	"\\n"
A literal carriage return character	␍	"\r"	\r	"\\r"
A literal double quote character, magic to Java, nothing special to regex.	"	"\""	"	"\""
A literal single quote character, magic to Java, nothing special to regex.	'	"\'"	'	"\'"
A literal backslash character, magic to both Java and regex.	\	"\\"	\\	"\\\\"

Regex Variations Table

I use three different regex engines many times a day. I have a heck of a time remembering which commands work with which one. So I composed this table. Lucky I don’t need Perl too.

Regex Variations Table
Use	Java 1.4+	SlickEdit® Unix	Funduc SR	Function
Use	Java 1.4+	SlickEdit Unix	Funduc SR	Function
search reserved	$ ( ) * + - . ? [ \ ] ^ { \| }	* + . ? \ { \| }	! $ ( ) * + - ? [ \ ] ^ \|	Reserved metachars in search strings must must be \-quoted if used as data chars. e.g. \+ \* \\|. If in doubt, quote. It won’t hurt.
replace reserved	\	\	% < > \	Reserved metachars in replace strings must must be \-quoted if used as data chars. e.g. \% \\ \< \> If in doubt, quote. It won’t hurt. In Java, you can abbreviate [a-z\.] as [a-z.] since . is clearly a character not a command inside [].
0+	greedy: * reluctant *?	*	*	Zero or More of the preceding thing. .* matches anything. In Funduc, the * comes before the thing repeated, e.g. [] to match anything even over multiple lines. In Java and SlickEdit, the comes after, e.g. [a-z]. Normally you want .?, the reluctant form instead of .* for wildcard matching.
1+	greedy: + reluctant +?	+	+	One or More of the preceding thing. In Funduc, the + comes before the thing repeated, e.g. +[0-9\,\.\+\-] to crudely match a number. In Java and SlickEdit, the + comes after, e.g. [0-9\,\.\+\-]+.
1	{1}	{1}	default	Exactly One of the preceding things, similarly for any {n}. Here is a cute trick to use this Java feature to count characters, inserting a dash between pairs of characters: // insert a dash between chars String cute = "AA54BG4G3G".replaceAll( "(\\w{2})(?!$)", "$1:" ); // cute is "AA-54-BG-4G-3G"
0 or 1	greedy: ? reluctant ??	?		Zero or One of the preceding thing. e.g (abc)? will match"" or "abc"
group	capturing: ( — ) non-capturing: (?: — )	( — )	( — )	Delimits a group of characters or patterns. The characters matching the group will show up when you call group(i). However, they won’t if you make the group non-capturing.
not char	^	~	?!()	Not character operator, e.g. In Java, [^abc] means anything but a, b or c. In other contexts ^ means start of line. In VSlick [~abc] means the same. In Funduc works only on expressions. xref?!(=) finds the letters xref followed by anything but =
not exp	(?!X)	~	!	Not expression operator. In java anything but X, via zero width negative lookahead. After the non-match, you continue where you left off, not at the end of the non-matching string. In Funduc xref?!(=) finds the letters xref followed by anything but =
or	\|	\|	\|	infix or Operator, (cat\|dog) matches cat or dog.
any	.	.	?	any char but newline. To make newline also match dot, in Java, embed (?s) early in the string. (?s) does not match anything, it just switches mode. You can also turn the mode on with a Pattern. compile flag DOTALL. You can turn it off again with (?-s).
nl	\r\n	\n	\r\n	newline, given for Windows.
sol	^	^	^	Start of Line. In other contexts means not. See notes on $.
eol	$	$	$	End of Line. For Windows, matchs a pair of characters \r\n. For Linux matches \n. For Mac matches \r. For Java, $ means end of string not end of line, unless you turn on multiline mode by embedding (?m) first. You can turn it off again with (?-m). This is subtle, and will drive you nuts. ^ and $ do not match any actual character. In multiline (?m) mode, they match an empty string right after or before a line terminator. Most of the time use (?s) and allow .*? to swallow newlines.
sof			^^	Start of File
eof			$$	End of File
range	[]	[]	[]	Range Operator, list of chars,[ab] means match a or b. [a-z] matches any character in range a through z. [0-9] is a digit. [a-z] is lower case. [A-Z] is upper case. [ -_] (space dash underscore) is any printable ASCII char. In Funduc, you don’t need parenthesis around [a-z] in the search string.
negation	[^, ]	[~, ]	n/a	any character except a comma or space
intersection	[a-z&&[^bc]]	n/a	n/a	a through z, except for b and c
sub	()	()	()	Sub-Expression. In Funduc, you don’t need parenthesis around [a-z] in the search string.
col			+n	Column Operator
replace	$1	\1 \2 etc.	%1 %2 etc. %1< (to lower case)	back reference to tagged expression #1, in () for replace. E.g. in SlickEdit to replace all occurrences of <span class="jmethod"> used before an upper case name, converting them to <span class="jclass"> . Search string : <span class="jmethod">([A-Z]) Replace string : <span class="jclass">\1 Remember to turn exact case matching on for these to work. In Funduc, you don’t need parenthesis around [a-z] in the search string. [a-z]* in Funduc will put the first character in %1 and the rest of the match in %2, very confusing. Java regex has only very primitive replace ability. Every match must be replaced by the same string. However, in Java you can also use \1 in the match string to insist on a match for some expression found earlier in the string, i.e. a repeated pattern.
replace example	search: \(([a-zA-z\(\"]) replace: \( \1	search: \(([a-zA-z\(\"]) replace: \( \1	search: \([a-zA-z\(\"] replace: \( %1	Replace all (x with ( x but only if x is alphabetic or ( or "
space	\s	[ \t\n]	[ \t\r\n]	single white space
spaces	\s+	\:b	+[ \t\r\n]	one or more white spaces, [ \t\n\x0B\f\r] Watch out, matches line end as well!
black	\S	[^ \t\n]	[! \t\r\n]	single non white space (blank, tab)
blacks	\S+	[^ \n\t]+	+[! \r\n\t]	one or more non-white spaces
word	(\p{Alpha}+)	\:w	+[A-Za-z]	alphabetic word (string of A-Z a-z )
number	([0-9\,\.\+\-]+)	([0-9\,\.\+\-]+)	+[0-9\,\.\+\-]	number (string of digits, commas, decimal points and signs)
quoted	\"(\\\"\|([ A-Za-z\'\[\]\+\=\!\@\# \$\%\^\&\* \<\>\:\;\?\\|\\]*))\"	\:q	\"(\\\"([ A-Za-z\'\[\]\+\=\!\@\# \$\%\^\&\ \<\>\:\;\?\\|\\]*))\"	quoted String. It easier just to quote all punctuation sometimes. It is easier to proofread. Don’t quote : in Vslick since \:… has special meaning.
special	\d = digit \D = non digit \s = single whitespace char \S = not whitespace \w = single alphanumeric char \W not alphanumeric \p{Lower} \p{Upper} \p{ASCII} \p{Alpha} \p{Digit} \p{Alnum} \p{Punct} \p{Graph} \p{Print} \p{Blank} \p{Cntrl} \p{XDigit} \p{Space} \p{Lu} \p{InGreek} \p{Sc} \P{InGreek} (?i) turn on case insensitive mode (?-i) turn on case sensitive mode	\:a alphanumeric char = [A-Za-z0-9] \:b blanks = ([ \t]+) \:c alpha char = [A-Za-z] \:d digit = [0-9] \:f filename part \:h hex = ([0-9A-Fa-f]+) \:i int = ([0-9]+) \:n float \:p path \:q quoted string \:v C language variable name = ([A-Za-z_$][A-Za-z0-9_$]*) \:w word = ([A-Za-z]+)		predefined match strings, e.g. \:w = ([A-Za-z]+) matches a word. Those are braces in \p{Alnum} not parentheses. It can be hard to tell in some typefaces. The strings are case sensitive, and when used in Java source code such strings must be coded as "\\p{Alnum}". Typically these abbreviations are not designed to work inside […].
capture	X{n,m} capturing / non-capturing constructs		%%srpath%% %%srfile%% %%srfiledate%% %%srfiletime%% %%srfilesize%% %%srdate%% %%srtime%% %%envvar=fruit%%	X{n,m} means X appears exactly n to m times.

This table only covers the most common magic characters. See the documentation for each Regex package for details.

Multiple Characters

Multiples in Java Regex
[A-Z]	A single upper-case letter
[A-Z]*	zero or more upper-case letters
[A-Z]+	one or more upper-case letters
[A-Z][A-Z]	Exactly two upper-case letters
[A-Z]{2}	Exactly two upper-case letters (same as above)
[A-Z]{2,}	Two or more upper-case letters
[A-Z]{2,10}	Between 2 and 10 (inclusive) upper-case letters
[a-zA-Z]	A single letter, upper- or lower-case

Awkward Characters

Here is how to represent various awkward characters. They represent the combined quoting needs for Java String literals and Regex Patterns.

How To Encode Awkward Characters
How	Desired
\\\\	\ The literal backslash character. You must double the \ twice since \ is the quoting character in both Java and Regex literals.
\\xhh	The character with hexadecimal value 0xhh, e.g. \\xff. Only works with two hex digits!
\uhhhh	The character with hexadecimal value 0xhhhh, e.g. \u20ac. Must always have exactly four hex digits. Don’t use for control characters e.g. 0..ff since \u expansion happens prior to compilation. In other words \u000a will start a new line in your program. Note there is only one lead \.
\\t	The tab character \u0009
\\n	The newline (line feed) character \u000a
\\r	The carriage-return character \u000d
\\f	The form-feed character \u000c
\\a	The alert (bell) character \u0007
\\e	The escape character \u001b
\\cx	control characters, e.g. \\cq for ctrl-q.
\\-	Literal -, not a regex range operator.
\\+	Literal +, not a regex operator.
\\*	Literal *, not a regex operator.
\\?	Literal ?, not a regex operator.
\\(	Literal (, not a regex expression bracketer.
\\)	Literal ), not a regex expression bracketer.
\\[	Literal [, not a regex expression bracketer.
\\]	Literal ], not a regex expression bracketer.
\\{	Literal {, not a regex expression bracketer.
\\}	Literal }, not a regex expression bracketer.
\\\|	Literal \|, not a regex operator.
\\$	Literal $, not a regex end of line.
\\^	Literal ^, not regex operator.
\\<	Literal <, not regex operator.
\\=	Literal =, not regex operator.

Terminology

Pattern.CASE_INSENSITIVE is a flag you can feed to Pattern.compile to do case insensitive searches. This is much easier than trying to do them directly in the regex strings.

Java 1.4.1+ regexes have assertions, extra conditions placed on the match. Colourful regex terminology includes:

capturing means characters are accumulated for Matcher.group.
greedy means find the last possible match (consuming the most text).
lookahead means it looks ahead for X.
lookbehind means it looks behind for X.
negative means the match fails if it finds X.
positive means the match succeeds if it finds X.
possessive means greedily match as much as you can and do not back off, even when doing so would allow the overall match to succeed. For example, if you applied the greedy regex .+ to abc you get abc.
reluctant means find the first possible match, If you applied the reluctant regex .+? to abc, you would just get a.
zero-width means it doesn’t actually capture any characters, or prevent them from being used in further matching.

The easiest way to understand these terms is to experiment with the various regex operators on simple strings. You can make yourself a test program that reads strings from the console. That way, at least you can avoid having to deal with Java \ string quoting. You only need concern yourself with regex \ quoting. You can also use the Quoter Amanuensis to first apply regex quoting then Java string quoting and let you paste the result into your program.

Pattern Flags

You can specifify flags to Pattern.compile( String regex, int flags) with:

By default regexes are case-sensitive.

Possible Pattern flags
Flag	Alternate Embedded Code	Notes
CASE_INSENSITIVE	(?i)	Makes case does not matter on matching, s matches S.
MULTILINE	(?m)	Make ^ and $ match embedded newlines. You might expect embedded newlines to match by default, but they don’t.
DOTALL	(?s)	Makes . match any character, including a line terminator. By default . does not match line terminators.
UNICODE_CASE	(?u)	Used in conjunction with CASE_INSENSITIVE to use the elaborate code-folding schemes to compare Unicode upper and lower case. By default, the presumption is all characters being matched are US-ASCI.
CANON_EQ		Treats canonically accented characters done with single char or with a pair as equivalent e.g . å : the pair "a\u030A" is the treated the same as the single character "\u00E5".
UNIX_LINES	(?d)	\n is recognised in ^ and $ processing.
LITERAL	\Q… \E	Treat all characters as ordinary literals rather than as commands. You don’t then quote with \.
COMMENTS	?x	Makes whitespace ignored, and allows embedded comments starting with # that are ignored until the end of a line.

Examples

The following examples use the Java conventions. For use on the command line, undouble the \\.

Negative Regex

Matching vs Finding vs LookingAt

Matching
When you want the entire String to match your Pattern, you use Matcher.matches.
Finding
When you want to find fragments in your String that match the Pattern, use Matcher.find.
If you only want to find the first occurence of a regex in a String you can use this simple method:
LookingAt
When you want to see if you String starts with your Pattern, use Matcher.lookingAt.

Splitting

Regexes can be used to break phrases into individual words. Here is an example:

Beware, split treats leading, embedded and trailing separators differently. It ignores trailing separators unless you use split ( string, -1 /* limit */ ). It inherited this oddity from Perl.

Tips

If you are having trouble composing a String to describe what you do want, try instead to compose one that describes what you don’t want, and reverse the sense of the match.
Don’t try to put all your logic in one be-all-end-all DEBE (Does Everything But Eat) regex. Use several simpler ones in succession. For example, if there are four distinct patterns you are looking for, use four regexes rather than one giant complicated one.
Always check the your regex to make sure you did not use an unquoted magic character as if it were an ordinary one. It is so easy to forget that a character is a command that you rarely use.
You will need some code like this if you want to include the separator character in your regexes, since \ has to be quoted in regexes.
Regexes are a sledgehammer for complex pattern matching. For simple tasks you can do the job at least three times faster and more simply with String. substring, String.indexOf, String. lastIndexOf, String. startsWith, String.endswWith, or possibly StringTokenizer or StreamTokenizer. Free free to mix regex logic with String method logic.
Regexes are not designed for complex language analysis like parsing XML or Java source code. Use a parser instead.
Compiling a Pattern is a non-trivial, time-consuming operation. If you are not carfeful, your Pattern will be recompiled on every use. For speed, use this idiom to compile the Pattern only once:
```
// ensuring the Pattern is compiled only once.
private static final Pattern p = Pattern.compile( "[a]*" );
```
Regexes rarely give av error message. They just fail to match anything. Start with just the first few chars of your regex and see what that matches. Then when that works and a few more characters at a time rather than trying to debug the whole thing at once. You can gradualy add characters to the right end of your regex, or gradually chop them off until it starts matching.
Matcher.group( i ) is full of surprises. Print out everything from 0 to n to make sure you are grabbing the right fragment.

String

The String class borrows some convenience regex methods, such as split, matches, replaceAll and replaceFirst. Normally you would use the more efficient java.util.regex methods such as Matcher.replaceFirst and Matcher.replaceAll where you precompile your Pattern and reuse it. The String versions are for one-shot use where efficiency is not a concern. Note that String. replace does not use regexes.

Books

recommend book⇒Mastering Regular Expressions, Powerful Techniques for Perl and Other Tools, Second Edition

paperback

ISBN13:

978-0-596-00289-3

ISBN10:

0-596-00289-0

publisher:

O’Reilly

published:

2002-07-15

by:

Jeffrey E. Friedl, Andy Oram

The Owl Book. Includes scripting languages such as Perl, Tcl, auk and Python. Does not specifically cover Java, though Java regexes were modeled on Perl. More a book for regex experts to hone their skills than a newbie to learn regexes. It is a good place to find regex solutions to standard problems. While it isn’t made up in cookbook style, the examples are usually real-life problems that can be put into practical use.

	abe books.co.uk	abe books.ca
	amazon.co.uk	amazon.ca
	abe books.de	chapters.indigo.ca
	amazon.de	abe books.com
	abe books.fr	amazon.com
	amazon.fr	barnes and noble.com
	abe books.it	powells.com
	iberlibro.com	o’reilly safari
		abe books anz

recommend book⇒Regular Expression Pocket Reference

paperback

ISBN13:

978-0-596-00415-6

ISBN10:

0-596-00415-X

publisher:

O’Reilly

published:

2003-05

by:

Tony Stubblebine

The Owl Cheat Sheet. Pocket reference companion to Mastering Regular Expressions which also has a owl on the cover.

	abe books.co.uk	abe books.ca
	amazon.co.uk	amazon.ca
	abe books.de	chapters.indigo.ca
	amazon.de	abe books.com
	abe books.fr	amazon.com
	amazon.fr	barnes and noble.com
	abe books.it	powells.com
	iberlibro.com	o’reilly safari
		abe books anz

Learning More

Sun’s tutorial on Regex

Sun’s tutorial on Greedy, Reluctant Possessive Quantifiers

Sun’s Javadoc on the Regex Package class : available:

on the web at java.sun.com
in the current JDK 1.6.0_10 or in the old JDK 1.5.0_16 on your local Windows J: drive.

Sun’s Javadoc on the Pattern class : available:

on the web at java.sun.com
in the current JDK 1.6.0_10 or in the old JDK 1.5.0_16 on your local Windows J: drive.

Sun’s Javadoc on the Matcher class : available:

on the web at java.sun.com
in the current JDK 1.6.0_10 or in the old JDK 1.5.0_16 on your local Windows J: drive.

Sun’s Javadoc on String.matches : available:

on the web at java.sun.com
in the current JDK 1.6.0_10 or in the old JDK 1.5.0_16 on your local Windows J: drive.

Sun’s Javadoc on String.replaceAll : available:

on the web at java.sun.com
in the current JDK 1.6.0_10 or in the old JDK 1.5.0_16 on your local Windows J: drive.

Sun’s Javadoc on String.replaceFirst : available:

on the web at java.sun.com
in the current JDK 1.6.0_10 or in the old JDK 1.5.0_16 on your local Windows J: drive.

Sun’s Javadoc on String.split : available:

on the web at java.sun.com
in the current JDK 1.6.0_10 or in the old JDK 1.5.0_16 on your local Windows J: drive.

Slick Edit documentation available from Help | contents ⇒ Search and Replace ⇒ Regular Expressions ⇒ Unix Regular Expressions.

Funduc search and replace documentation is available from Help ⇒ contents ⇒ Regular Expressions | Search Operators.

4NT documentation is available from help | contents ⇒ wildcards ⇒ advanced wildcards

Apache RegExp
Eclipse Regex tester
Expresso Regex tester: for Windows
finite state automaton
JavaRegex.com
JFlex
JRegex
KRegExpEditor: part of KDE utilities
list of regex parsers
P~: a Java-like algebraic regex syntax
QuickRex Regex tester
Quoter Regex Amanuensis
Regex Buddy: a tool to let you compose regexes by picking items from a menu
Regex Composer student project
Regex Debugger student project
Regex Proofreader student project
regex tutorial
regular-expressions.info
Sample Regexes (.net style)
Savarese OroMatcher
Scanner
StreamTokenizer
String
StringTokenizer

	Please email your feedback for publication, errors, omissions, broken/redirected link reports and suggestions to improve this page to Roedy Green :
	Canadian Mind Products
	mindprod.com IP:[65.110.21.43]
	Your face IP:[38.103.63.62]	The information on this page is for non-military use only.
	You are visitor number 128,545.	Military use includes use by defence contractors.
	You can get a fresh copy of this page from:	or possibly from your local J: drive (Java virtual drive/mindprod.com website mirror)
	http://mindprod.com/jgloss/regex.html	J:\mindprod\jgloss\regex.html