The need for expression evaluation is popular in larger and more complex applications, especially when you want to give your users (or people that configure it) some flexibility in terms of the expected behavior of the program. Imagine a budgeting app, which is supposed to categorize your expenses based on the place you made a purchase, and its value. Some people would include $5 chocolate bar as just a part of their regular diet, while others would put it as a whim, that could (and should) be avoided. You can probably imagine that the configuration possibilities are limitless, and Cel language gives you the ability to parametrize your app as much as possible here.

What is cel-go

The Common Expression Language (CEL) is a non-Turing complete language designed for simplicity, speed, safety, and portability. CEL's C-like syntax looks nearly identical to equivalent expressions in C++, Go, Java, and TypeScript. (source)

Building blocks

To start working with cel, you should first understand what are the elements that this powerful tool consists of. First of all, you need to set up an environment that keeps all the definitions used during your program execution (and expression evaluation). This includes custom types (primitives are a part of the defaults) and variable declarations (ie. the names that you can use with their types), function definitions (names with arguments, their types, and the result type).

env, err := cel.NewEnv(
    cel.Types(/* custom types */),
    cel.Declarations(
        decls.NewVar("thing", /* type of variable 'thing' */),
        decls.NewFunction("is_true",
            decls.NewInstanceOverload("is_true",
                []*exprpb.Type{/* types of arguments */},
                decls.Bool /* return type */,
            ),
        ),
        ...
    ),
)

Then, you need to provide all the expressions to your environment as string values, so that you can compile them to see if they are manageable by your program:

ast, issues := env.Compile(rule)
// issues consist of any compilation errors that occurred

The compilation results in an abstract syntax tree (therefore the variable name, ast) and can be passed further to the program phase:

program, err := env.Program(ast, options)

This step optionally accepts options that modify the execution. While you may not use the globals (cel.Globals) too often, you should definitely get familiar with cel.Functions, where you need (if you choose to use them) provide an implementation of all functions that you have previously declared:

funcs := cel.Functions(
    &functions.Overload{
        Operator: "is_true",
        Binary: func(lhs, rhs ref.Val) ref.Val {
            ...
            return types.Bool(true)
        },
    },
    ...
)

Finally, with all this ready and waiting, you can evaluate input parameters against the rules and see the final output. All you have to do at this point is execute Eval function on your program with the map of variables and their values:

val, _, err := program.Eval(map[string]interface{}{
    "thing": "a table",
    "number": 11,
})
...
bVal, ok := val.Value().(bool) // given that your expression returns a boolean value
if ok {
    return bVal
}

Looks complicated? Let's take it one step at the time.

Basic example

The first thing you may want to do when building an expression parser is start with operating on the primitive types, and utilizing the features that come built-in the language. Let's build a simple "phone number verifier", that would be able to decide if an input value is a valid number. The rules, however, are different for specific countries, so that you want to define them separately.

For example, a valid Polish mobile phone number has nine digits. This does not include a country prefix, which can be either one of "+48" and "0048". This way we can create a set of rules as:

  • it has nine digits
  • it has 11 digits and starts with "+48"
  • it has 13 digits and starts with "0048"

Then, we want to translate it to the expression language, so the rules would look like this (with input being, obviously, an input value):

  • has_digits(number, 9) && has_length(number, 9)
  • has_digits(number, 11) && number.size() == 12 && number.starts_with('+48')
  • has_digits(number, 13) && number.size() == 13 && number.starts_with('0048')

Note, that has_digits takes number as the first parameter, while starts_with is an instance function just to showcase different ways of creating functions in Cel. You can make both of them of the same "type", if that makes more sense to you. The third one, size(), is a built-in in Cel's standard library.

Custom function

In plain Go, we can define two types of functions:

// A regular function that takes in all parameters:
func add(a, b int) int {...}

// An instance function where one of the parameters is the instance on which we perform action:
func (n *Value) change(property, value string) {...}

In cel we can do the same, and we define each function in two steps - by defining its declaration in the env, and by providing its definition (aka overload).

Regular function

The declaration uses decls.NewOverload(..) as the only argument, which defines the name, a list of argument types, and a return type:

// A regular function - declaration
env, err := cel.NewEnv(
        ...
        decls.NewFunction("has_digits",
            decls.NewOverload("has_digits",
                []*exprpb.Type{decls.String, decls.Int},
                decls.Bool,
            ),
        ),
    ),
)

The actual implementation depends on the number of arguments, and since we have two of them, we need to provide the function body as Binary field of the *functions.Overload structure. Since the field is pretty generic, you need to cast these ref.Val values into the types that you want to operate on:

// A regular function - overload
funcs := cel.Functions(
    ...
    &functions.Overload{
        Operator: "has_digits",
        Binary: func(lhs, rhs ref.Val) ref.Val {
            val := lhs.(types.String)
            strVal := val.Value().(string)

            length := rhs.(types.Int)
            intLength := length.Value().(int64)

            runes := []rune(strVal)

            var digits int64 = 0
            for _, r := range runes {
                if unicode.IsDigit(r) {
                    digits += 1
                }
            }

            return types.Bool(digits == intLength)
        },
    },
)

A regular function is used the same way as in Go, ie. has_digits(arg1, arg2).

Instance function

An instance function is not that different from the regular one, as the arguments are defined as the exact same list. The thing that you need to change here is that now we are using decls.NewInstanceOverload(..) that suggests that the first argument is the instance on which we will be calling the function:

// An instance function - declaration
env, err := cel.NewEnv(
    cel.Declarations(
        ...
        decls.NewFunction("starts_with",
            decls.NewInstanceOverload("starts_with",
                []*exprpb.Type{decls.String, decls.String},
                decls.Bool,
            ), 
        ),
    ),
)

The overload does not differ from the regular function - since there are two arguments we are still using Binary function to define the action we want to perform:

// An instance function - overload
funcs := cel.Functions(
    &functions.Overload{
        Operator: "starts_with",
        Binary: func(lhs, rhs ref.Val) ref.Val {
            val := lhs.(types.String)
            strVal := val.Value().(string)

            prefix := rhs.(types.String)
            strPrefix := prefix.Value().(string)

            return types.Bool(strings.HasPrefix(strVal, strPrefix))
        },
    },
)

An instance function is used the same way as in Go, ie. arg1.starts_with(arg2).

Defining custom types

Until now, we limited ourselves to the primitives, but any complex application at some point would require us to define some custom types and use them in the expressions as well. While there is basically no limitation of the custom types we can define and use, cel requires us to define them using protocol buffers. In the example below, we would need two structures:

// models/models.proto
...
package mycodesmells.celgo.models;
...

message Person {
    string name = 1;
    string country = 2;
    int32 age = 3;
}

message Country {
    string code = 1;
    bool beer_legal = 2;
    int32 beer_age_limit = 3;
}

Once we generate a source code file (protoc --go_out=${GOPATH}/src models/models.proto) we are good to go with including these types in our expression language.

To make the types recognized by our cel environment, we need to add the types:

env, err := cel.NewEnv(
    cel.Types(&models.Person{}, &models.Country{}),
    cel.Declarations(
        decls.NewVar("person", decls.NewObjectType("mycodesmells.celgo.models.Person")),
        decls.NewVar("countries", decls.NewListType(decls.NewObjectType("mycodesmells.celgo.models.Country"))),
...

Also, each declaration show which variable is of what type. For example, a person is of type Person, because the argument passed to decls.NewObjectType is a concatenation of package name (defined in .proto file in the top) and the type name (define in the very same file, as the message name).

If we need to return an instance of one of our types from the function, we also need a thing called type registry defined as follows:

reg := types.NewRegistry(
    &models.Country{},
    &models.Person{},
)

This is necessary because the return data must be of ref.Val instance, and having the registry and an actual value we can fulfill these requirements with a smart helper function:

c := &models.Country{...}
v := reg.NativeToValue(c) // v is of type ref.Val

Using custom in functions

Using primitives was simpler because cel library provided us with the type representations, and the casting did not require us to do any unusual hacks. We could cast text values to types.String and we could focus on the core functionality. How does it look with custom types? Fortunately, there is a way to do this as well, thanks to the reflect package from Go's standard library:

// raw is of type ref.Val
x, _ := raw.ConvertToNative(reflect.TypeOf(&models.Person{}))
person := x.(*models.Person)
// person is of type *models.Person and ready to use

Apart from these differences, the functions using custom types as both instance, and regular input parameters work the same way as if the values were primitives:

// returns if a person's age is larger than a given number
env, err := cel.NewEnv(
    cel.Types(&models.Person{}, &models.Country{}),
    cel.Declarations(
        ...
        decls.NewFunction("older_than", 
            decls.NewInstanceOverload("older_than",
                []*exprpb.Type{decls.NewObjectType("mycodesmells.celgo.models.Person"), decls.Int},
                decls.Bool,
            ),
        ),
        // returns if a person's age is large enough to be able to buy a beer in a given country
        decls.NewFunction("meets_age_limit",
            decls.NewOverload("meets_age_limit",
                []*exprpb.Type{
                    decls.NewObjectType("mycodesmells.celgo.models.Person"),
                    decls.NewObjectType("mycodesmells.celgo.models.Country"),
                },
                decls.Bool,
            ),
        ),
        // returns a country definition from a person's country code
        decls.NewFunction("country",
            decls.NewOverload("country",
                []*exprpb.Type{
                    decls.NewListType(decls.NewObjectType("mycodesmells.celgo.models.Country")),
                    decls.NewObjectType("mycodesmells.celgo.models.Person"),
                },
                decls.NewObjectType("mycodesmells.celgo.models.Country"),
            ),
        ),

funcs := cel.Functions(
    &functions.Overload{
        Operator: "older_than",
        Binary: func(lhs, rhs ref.Val) ref.Val {
            x, _ := lhs.ConvertToNative(reflect.TypeOf(&models.Person{}))
            person := x.(*models.Person)

            ageTyped := rhs.(types.Int)
            age := ageTyped.Value().(int64)

            return types.Bool(int64(person.Age) >= age)
        },
    },
    &functions.Overload{
        Operator: "meets_age_limit",
        Binary: func(lhs, rhs ref.Val) ref.Val {
            x, _ := lhs.ConvertToNative(reflect.TypeOf(&models.Person{}))
            person := x.(*models.Person)

            y, _ := rhs.ConvertToNative(reflect.TypeOf(&models.Country{}))
            country := y.(*models.Country)

            return types.Bool(person.Age > country.BeerAgeLimit)
        },
    },
    &functions.Overload{
        Operator: "country",
        Binary: func(lhs, rhs ref.Val) ref.Val {
            x, _ := lhs.ConvertToNative(reflect.TypeOf([]*models.Country{}))
            countries := x.([]*models.Country)

            y, _ := rhs.ConvertToNative(reflect.TypeOf(&models.Person{}))
            person := y.(*models.Person)
            countryCode := person.Country

            for _, country := range countries {
                if country.Code == countryCode {
                    return reg.NativeToValue(country)
                }
            }

            return nil
        },
    },

Real-life scenario

To show that all this makes sense, I prepared a simple beer bar example. The bar hires a few bartenders/waiters that have a different approach of serving beer to the customers. Some check your ID, some don't, and some have additional quirks that affect their decision. For example, one of the waiters won't sell a beer to anyone below 30, as his call is a result of person.older_than(30) && meets_age_limit(person, country(countries, person)).

With different customers coming in, you can see how the decision varies between the waiters. Feel free to checkout the code and play around to see how you can get something to drink in this crazy place!

Summary

While cel can feel like an overwhelming tool, and definitely it is quite powerful. The key to learning how to use it is taking your time, and learning it step by step, steadily expanding your understanding of what can be done and how. Remember, that you can eat an elephant one bite at a time!

Links