Programming

Open Sourced my JavaScript Regular Expression Generator - RegexGen.js


Hi there, I've open-sourced my new library, RegexGen.js, a JavaScript regular expression generator, please give it a try. Comments and issue reports are welcome. Thank you!

RegexGen.js - JavaScript Regular Expression Generator

RegexGen.js is a JavaScript regular expression generator that helps to construct complex regular expressions, inspired by JSVerbalExpressions.

RegexGen.js is basically designed for people who know how the regular expression engine works, but not working with it regularly, i.e., they know how to make the regex works but may not remember every meta-characters that constructs the regex.

RegexGen.js helps people don't have to remember: meta-characters, shortcuts, what characters to escape and tricks about corner cases (http://stackoverflow.com/questions/5484084/what-literal-characters-should-be-escaped-in-a-regex/5484178#5484178).

RegexGen.js helps reusing regex patterns. (checkout the [Matching an IP Address] example bellow.)

Fork me on GitHub

The Problems

RegexGen.js tries to ease two problems.

  1. While creating a regular expression, it's hard to remember the correct syntax and what characters to escape.
  2. After done creating a regular expression, it's hard to read and remember what the regex do.

The Goals

RegexGen.js is designed to achieve the following goals.

  1. The written codes should be easy to read and easy to understand.
  2. The generated code should be as compact as possible, e.g., no redundant brackets and parentheses.
  3. No more character escaping reguired (except '\', or if you use regex overwrite.)
  4. If the generated code is not good enougth, bad parts can be easily replaced directly in the written codes.

Getting Started

The generator is exported as a regexGen() function.

To generate a regular expression, pass sub-expressions as parameters to the call of regexGen() function.

Sub-expressions which are separated by comma are concatenated together to form the whole regular expression.

Sub-expressions can either be a string, a number, a RegExp object, or any combinations of the call to methods (i.e., the sub-generators) of the regexGen() function object, as the following informal BNF syntax.

Strings passed to the the call of regexGen(), text(), maybe(), anyCharOf() and anyCharBut() methods, are always escaped as necessary, so you don't have to worry about which characters to escape.

The result of calling the regexGen() function is a RegExp object.

1
2
var regexGen = require('regexgen.js');
var regex = regexGen( sub-expression [, sub-expression ...] [, modifier ...] )

The basic usage can be expressed as the following informal BNF syntax.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
regex ::= regexGen( sub-expression [, sub-expression ...] [, modifier ...] )

sub-expression ::= string | number | RegExp object | term

term ::= sub-generator() [.term-quantifier()] [.term-lookahead()]

sub-generator() ::= regexGen.startOfLine() | regexGen.endOfLine()
    | regexGen.wordBoundary() | regexGen.nonWordBoundary()
    | regexGen.text() | regexGen.maybe() | regexGen.anyChar() | regexGen.anyCharOf() | regexGen.anyCharBut()
    | regexGen.either() | regexGen.group() | regexGen.capture() | regexGen.sameAs()
    | regex() | ... (checkout wiki for all sub-generators.)

term-quantifier() ::= .term-quantifier-generator() [.term-quantifier-modifier()]

term-quantifier-generator() ::= term.any() | term.many() | term.maybe() | term.repeat() | term.multiple()

term-quantifier-modifier() ::= term.greedy() | term.lazy() | term.reluctant()

term-lookahead() ::= term.contains() | term.notContains() | term.followedBy() | term.notFollowedBy()

modifier ::= regexGen.ignoreCase() | regexGen.searchAll() | regexGen.searchMultiLine()

Please check out regexgen.js and wiki for API documentations, and check out test.js for more examples.

Installation

npm install regexgen.js

Usage

Since the generator is exported as the regexGen() function, everything must be referenced from it.
To simplify codes, assign it to a short variable is preferable.

1
2
3
4
5
6
7
8
9
10
var  = require('regexgen.js');
var regex = (
    .startOfLine(),
    .capture( 'http', .maybe( 's' ) ), '://',
    .capture( .anyCharBut( ':/' ).repeat() ),
    .group( ':', .capture( .digital().multiple(2,4) ) ).maybe(), '/',
    .capture( .anything() ),
    _.endOfLine()
);
var matches = regex.exec( url );

Note: Though not recommended, if you still feel inconvenient, and don't mind the global object being polluted,
use the regexGen.mixin() function to export all member functions of the regexGen() function object to the global object.

1
2
3
4
5
6
7
8
9
10
11
12
var regexGen = require('regexgen.js');
regexGen.mixin( global );

var regex = regexGen(
    startOfLine(),
    capture( 'http', maybe( 's' ) ), '://',
    capture( anyCharBut( ':/' ).repeat() ),
    group( ':', capture( digital().multiple(2,4) ) ).maybe(), '/',
    capture( anything() ),
    endOfLine()
);
var matches = regex.exec( url );

About The Returned RegExp Object

The RegExp object returned from the call of regexGen() function, can be used directly as usual.
In addition, there are four properties injected to the RegExp object:
warnings array, captures array, extract() method and replace() method.
Checkout wiki for details.

Examples

Simple Password Validation

This example is taken from the article: Mastering Lookahead and Lookbehind.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
var  = require('regexgen.js');
var regex = (
    // Anchor: the beginning of the string
    .startOfLine(),
    // Match: six to ten word characters
    .word().multiple(6,10).
        // Look ahead: anything, then a lower-case letter
        contains( .anything().reluctant(), .anyCharOf(['a','z']) ).
        // Look ahead: anything, then an upper-case letter
        contains( .anything().reluctant(), .anyCharOf(['A','Z']) ).
        // Look ahead: anything, then one digit
        contains( .anything().reluctant(), .digital() ),
    // Anchor: the end of the string
    _.endOfLine()
);

Generates:

1
/^(?=.?[a-z])(?=.?[A-Z])(?=.*?\d)\w{6,10}$/

Matching an IP Address

This example is taken from the book: Mastering Regular Expressions

1
2
3
4
5
6
7
8
9
10
var  = require('regexgen.js');
var d1 = .group( .anyCharOf( '0', '1' ).maybe(), .digital(), .digital().maybe() );
var d2 = .group( '2', .anyCharOf( ['0', '4'] ), .digital() );
var d3 = .group( '25', .anyCharOf( ['0', '5'] ) );
var d255 = .capture( .either( d1, d2, d3 ) );
var regex = (
    .startOfLine(),
    d255, '.', d255, '.', d255, '.', d255,
    _.endOfLine()
);

Generates:

1
/^([01]?\d\d?|2[0-4]\d|25[0-5]).([01]?\d\d?|2[0-4]\d|25[0-5]).([01]?\d\d?|2[0-4]\d|25[0-5]).([01]?\d\d?|2[0-4]\d|25[0-5])$/

Matching Balanced Sets of Parentheses

This example is taken from the book: Mastering Regular Expressions

1
2
3
4
5
6
7
8
9
10
11
12
var  = require('regexgen.js');
var regex = (
    '(',
    .anyCharBut( '()' ).any(),
    .group(
        '(',
        .anyCharBut( '()' ).any(),
        ')',
        .anyCharBut( '()' ).any()
    ).any(),
    ')'
);

Generates:

1
/([^()](?:([^()])[^()]))/

Matching Balanced Sets of Parentheses within Any Given Levels of Depth

This example is taken from the book: Mastering Regular Expressions

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
var  = require('regexgen.js');
function nestingParentheses( level ) {
    if ( level < 0 ) {
        return '';
    }
    if ( level === 0 ) {
        return .anyCharBut( '()' ).any();
    }
    return .either(
            .anyCharBut( '()' ),
            _.group(
                '(',
                nestingParentheses( level - 1 ),
                ')'
            )
        ).any();
}

Given 1 level of nesting:

1
2
3
var regex = _(
    '(', nestingParentheses( 1 ), ')'
);

Generates:

1
/((?:[^()]|([^()])))/

Given 3 levels of nesting:

1
2
3
var regex = _(
    '(', nestingParentheses( 3 ), ')'
);

Generates:

1
/((?:[^()]|((?:[^()]|((?:[^()]|([^()])))))))/

Matching an HTML Tag

This example is taken from the book: Mastering Regular Expressions

1
2
3
4
5
6
7
8
9
10
var  = require('regexgen.js');
var regex = (
    '<',
    .either(
        .group( '"', .anyCharBut('"').any(), '"' ),
        .group( "'", .anyCharBut("'").any(), "'" ),
        .group( _.anyCharBut( '"', "'", '>' ) )
    ).any(),
    '>'
);

Generates:

1
/<(?:"[^"]"|'[^']'|[^"'>])*>/

This example is taken from the book: Mastering Regular Expressions

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
var  = require('regexgen.js');
var regexLink = (
    '<a',
    .wordBoundary(),
    .capture(
        .anyCharBut( '>' ).many()
    ),
    '>',
    .capture(
        .label( 'Link' ),
        .anything().lazy()
    ),
    '</a>',
    .ignoreCase(),
    .searchAll()
);
var regexUrl = (
    .wordBoundary(),
    'href',
    .space().any(), '=', .space().any(),
    .either(
        .group( '"', .capture( .anyCharBut( '"' ).any() ), '"' ),
        .group( "'", .capture( .anyCharBut( "'" ).any() ), "'" ),
        .capture( .anyCharBut( "'", '"', '>', .space() ).many() )
    ),
    _.ignoreCase()
);

Generates:

1
2
/<a\b([^>]+)>(.?)<\/a>/gi
/\bhref\s=\s(?:"([^"])"|'([^']*)'|([^'">\s]+))/i

Here's how to iterate all links (in browser):

1
2
3
4
5
6
7
8
9
var capture, guts, link, url, html = document.documentElement.outerHTML;
while ( (capture = regexLink.exec( html )) ) {
    guts = capture[ 1 ];
    link = capture[ 2 ];
    if ( (capture = regexUrl.exec( guts )) ) {
        url = capture[ 1 ] || capture[ 2 ] || capture[ 3 ];
    }
    console.log( url + ' with link text: ' + link );
}

Examining an HTTP URL

This example is taken from the book: Mastering Regular Expressions

1
2
3
4
5
6
7
8
9
var  = require('regexgen.js');
var regex = (
    .startOfLine(),
    'http', .maybe( 's' ), '://',
    .capture( .anyCharBut( '/:' ).many() ),
    .group( ':', .capture( .digital().many() ) ).maybe(),
    .capture( '/', .anything() ).maybe(),
    .endOfLine()
);

Generates:

1
/^https?:\/\/([^/:]+)(?::(\d+))?(\/.*)?$/

Here's a snippet to report about a URL (in browser):

1
2
3
4
5
var capture = location.href.match( regex );
var host = capture[1];
var port = capture[2] || 80;
var path = capture[3] || '/';
console.log( 'host:' + host + ', port:' + port + ', path:' + path );

Validating a Hostname

This example is taken from the book: Mastering Regular Expressions

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
var  = require('regexgen.js');
var regex = (
    .startOfLine(),
    // One or more dot-separated parts . . .
    .either(
        .group(
            .anyCharOf( ['a', 'z'], ['0', '9'] ),
            '.'
        ),
        .group(
            .anyCharOf( ['a', 'z'], ['0', '9'] ),
            .anyCharOf( '-', ['a', 'z'], ['0', '9'] ).multiple( 0, 61 ),
            .anyCharOf( ['a', 'z'], ['0', '9'] ),
            '.'
        )
    ).any(),
    // Followed by the final suffix part . . .
    .either(
        'com', 'edu', 'gov', 'int', 'mil', 'net', 'org', 'biz', 'info', 'name', 'museum', 'coop', 'aero',
        .group( .anyCharOf( ['a', 'z'] ), .anyCharOf( ['a', 'z'] ) )
    ),
    _.endOfLine()
);

Generates:

1
/^(?:[a-z0-9].|[a-z0-9][-a-z0-9]{0,61}[a-z0-9].)*(?:com|edu|gov|int|mil|net|org|biz|info|name|museum|coop|aero|[a-z][a-z])$/

Parsing CSV Files

This example is taken from the book: Mastering Regular Expressions

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
var  = require('regexgen.js');
var regex = (
    .either( .startOfLine(), ',' ),
    .either(
        // Either a double-quoted field (with "" for each ")
        .group(
            // double-quoted field's opening quote
            '"',
            .capture(
                .anyCharBut( '"' ).any(),
                .group(
                    '""',
                    .anyCharBut( '"' ).any()
                ).any()
            ),
            // double-quoted field's closing quote
            '"'
        ),
        // Or some non-quote/non-comma text....
        .capture(
            .anyCharBut( '",' ).any()
        )
    )
);

Generates:

1
/(?:^|,)(?:"([^"](?:""[^"]))"|([^",]))/

Change logs

2015-09-12:

  • Remove UMD headers that support RequireJS and browser globals.
  • The json object returned from extract() method without the "0" property.
  • Add replace() method to RegExp object.

2014-09-20:

  • Fix CommonJS factory invoking bug.

2014-08-17:

  • Rename RegExp.jsonExec() to extract().

2014-08-15:

  • Character Classes now support nesting.
  • Fix a bug in multiple(). In the case of multiple(5) that returns /{,}/, and should be /{5,}/.

2014-08-10:

  • Added RegExp.jsonExec() method, that returns a JSON object using capture names as properties.

Author

References