Mini Go Apps

Prequel

Over the years, I have had requests for small, dedicated focused apps. Since I started with Python, I would write them in Python. For building the app itself, this worked fine. However, when I needed to share or deploy the app, I had to make sure they could even run it. If I dared to use any extensions, then the person running it also needed to install those extensions.

I looked into compiling Python code into an executable, with mixed results. I could never really be happy with it because it felt hackish and counter to how you’re supposed to use Python. Plus, the binaries would get flagged as malware or spam.

This is a very un/under documented part of Python. - reddit

Compiling with Go

This is one of the main perks to using Go (Golang). I can compile the code, get a binary for multiple architectures and share the binary they need. For example, here is a minimal go app:

package main

import (
    "fmt"
)

func main() {
    fmt.Println("Hello World")
}

Now we’ll compile it for windows. We’re building on a Linux machine, so we need to cross-compile.

export GOOS="windows"
export GOARCH="amd64"
go build hello_world.go -o hello_world.exe

Drop it into Windows and we can run it!

hello_world.exe
Hello World

You’ll need to take into consideration differences of OS’s, such as where to store configuration files. However, this is otherwise a very easy way to get applications to users with other operating systems. They don’t need to install anything, or extensions, just take your binary and run it.

Mini Go Apps

This will be a multipart post, with each post breaking down how we’re going to make a few mini Go apps. At the top of each post (and right here), you will be able to navigate between each one.

For the client applications, we will be relying on the CLI (Command Line Interface) to interact with it. We will use flags to change the behavior.

For server applications, we will be building them using 12 Factor principles. I do not plan on using any external libraries. But to expand upon, CLI libraries like cobra are excellent.

Don’t worry however! We will be keeping these apps fairly small. For CLI apps, their primary focus will be to get or push data, and do something with the result.

For server apps, their focus will be to provide a JSON response in some way that is useful.

Let’s get Started

Slight detour (0-50 hours). If you don’t have Go already installed, here is the official guide.

If you’ve never used Go, it is wise to get a decent overview. It will give you context and a background. While will I try to have a straightforward guide, having that background will let you get a lot more out of these posts. This is what I used to get started (non-affliate).

Course	Format	Benefits
Go: The Complete Developers Guide	Videos	Absolute Beginner, receivers, data types.
Let’s Go	Book	Structuring an app, API, databases, testing.
Ultimate Go	Videos	Go principles, Underlying data structures, pointers.

Go is best used with an IDE, but not necessary for these posts. If you want suggestions:

Visual Studio Code (Free)
Jetbrains Goland (Paid)

Ok, good? Moving on.

Our first application is very simple. It will take in information as stdin, check to see if that input matches a regex and return the matched target as stdout if it does. If it doesn’t, it will return an error code of 1.

We’ll jump into the boilerplate. Notice it’s nearly similar to the hello world program, but we’ve added in some (c) comments. Once we’ve done these todo’s, we should have a functioning app!

package main

import (
    "fmt"
)

func main() {
    fmt.Println("Hello World")

    // TODO Validate and create regex
    // TODO Get stdin input
    // TODO Check to see if it matches regex
    // TODO Output matched string
}

Don’t bother compiling this just yet, (it’s the same as the hello world app), let’s just start with the second TODO: Get stdin input.

How do we do this? I don’t know. I haven’t memorized go’s syntax for i/o. So instead of just telling you the answer, let’s figure it out together.

Getting input

I need to get input, from stdin. We’re working in Go. Let’s search for that: duckduckgo: golang get stdin. Stack overflow has a post, we’ll check that.

This answer has an example. Looking at (1), I notice two things:

bufio is used, reading from os.Stdin
It uses ReadString

reader := bufio.NewReader(os.Stdin)
fmt.Print("Enter text: ")
text, _ := reader.ReadString('\n')
fmt.Println(text)

Great. Lets checkout Go’s standard library and search for bufio. Specifically the ReadString part.

Alright, it essentially returns a string and error of the delimiter. Since we’re using '\n', it reads one line and returns. We now have a design decision. Suppose we provide an input like "a\nb\nc", which is a, b, c on different lines. Should our program just check a? Or should it check a b c? Or should there be a flag to configure this behavior?

In this case, we’re going to start simple and just check the first line. But take a moment to consider how you’d handle those other circumstances. Looping over the reader could work, and using flags for configuring behavior might work too. We’ll revisit this.

We’ll use those lines from above into our program:

package main

import (
    "fmt";
    "bufio";
    "os";
)

func main() {
    fmt.Println("Hello World")

    // TODO Validate and create regex
    // TODO Get stdin input
    reader := bufio.NewReader(os.Stdin)
    text, _ := reader.ReadString('\n')
    // TODO Check to see if it matches regex
    // TODO Output matched string

    fmt.Println(text)
}

We can test and run this in the shell.

$ go build -o matcher main.go
$ echo "hi" | ./matcher
hi

Notice we’re building an output file of matcher. To wrap this into one step, you can also do this while developing:

echo "hi" | go run main.go
hi

Great! We input a line through stdin, and it echoed it right out. Now let’s do something with it when we run our program.

Defining and using Regex

Regex (Regular Expressions) is a semi-standardized way of matching for text. While it is worthwhile searching for how to learn regex The truth is I consider regex to be a write but do not read language. That is, once you’ve written it, don’t expect to be able to read it. And for writing it, I’ll use regex101, get something that matches my use case and move on.

To start, let’s try to pattern match a hard coded expression of something that starts with a number, then 2 letters. Here we go:

\d\w{2}

Let’s look at the standard library again for regexp. I don’t know how to define a pattern, or how to use it. I’m hoping to find answers here. I can also just search how to use regex with golang.

In the regexp library, Match looks promising, and the example looks like what I want. Sort of. I don’t really want a true/false. I want the actual string it matched on.

matched, err := regexp.Match(`foo.*`, []byte(`seafood`))
fmt.Println(matched, err)

true <nil>

Also, since I may be reusing the regex expression multiple times, I also want to compile it and use that directly. I noticed Compile and MustCompile. I notice these return a *Regexp as well. In their website, blue objects are links we can use to route to that object. So let’s look at the Regex object.

This just led to a type, which feels like a dead end. Going back, I discover Find and its output is the matched string. The example given also shows a pre-compiled regex. Great! We’ll use this:

re := regexp.MustCompile(`foo.?`)
fmt.Printf("%q\n", re.Find([]byte(`seafood fool`)))

We’ll keep this part in mind too:

A return value of nil indicates no match.

Finally, how do we exit out of a program with a specific error code? Since we’re sending the signal to the operating system, let’s look at os. Exit looks promising. 0 indicates no error, so os.Exit(1) should work fine for us.

Notice in the code below the (r) regex specific lines where we are hardcoding the regex pattern. For (o), remember a design pattern we decided for this program:

check to see if that input matches a regex and return the matched target as stdout if it does. If it doesn’t, it will return an error code of 1.

package main

import (
    "fmt";
    "bufio";
    "os";
    "regexp";
)

func main() {
    // TODO Validate and create regex
    re := regexp.MustCompile(`\d\w{2}`)
    // TODO Get stdin input
    reader := bufio.NewReader(os.Stdin)
    text, _ := reader.ReadString('\n')
    // TODO Check to see if it matches regex
    m := re.Find([]byte(text))
    // TODO Output matched string
    if m == nil {
        os.Exit(1)
    } else {
        fmt.Print(text)
    }
}

Looks good, so we’ll test it now.

> echo hi | go run main.go
exit status 1
> echo 1bc | go run main.go
1bc

Odd! Why is it returning the string exit status 1? We know why it is exiting with an exit code of 1, but not why it’s showing that string!

Searching golang printing exit status 1 shows some results. We dive into a Google Groups, which leads to a GitHub issue. Dropping into an apparently heated battle about go run, well, we can notice it might have to do with go run versus running a built binary instead.

Let’s test it.

$ go build -o matcher main.go
echo hi | ./matcher
echo 1abc | ./matcher
1abc

Solved! Now we have another choice in front of us. Do we no longer use go run and instead build it before running each time, or do we ignore the different output of using go run?

Up to you. My thinking is this: Since the behavior has changed somewhat, we might run into unexpected issues now. Since the change to compiling before and running is trivial, let’s do that instead.

$ go build -o matcher main.go && echo hi | ./matcher
go build -o matcher main.go && echo 1abc | ./matcher
1abc

Command runner detour (optional)

Now our command is starting to look a little complex. We can keep going down this path (which is fine!), or we can use a command runner. An easy example would be using a bash script. I am partial to using just. Here is examples for each, where you can run each, respectfully:

./run.sh
just run

go build -o matcher main.go
echo hi | ./matcher
echo 1cb | ./matcher

run:
    go build -o matcher main.go
    echo hi | ./matcher
    echo 1cb | ./matcher

These actually look like test cases, something Go is great for. But we’ll visit that a bit later.

Configure with flags

Time to get out of hard coding the regex. We’re going to define it each time we run the program. We have a number of different ways to do this.

Configuration file (local or on the network)
Environment variables
Flags

We’re going to use flags. So, our goal is to have something like this:

echo 1cd | ./matcher -e '\d\w{2}'
1cd

Since we’re enjoying the standard library documentation so much, let’s look for flags there. We’re going to the use flag package. As a side note, the pflag, while not standard, is excellent.

Let’s look at our TODO. We need to collect the provided regex string, validate it and create it. The second goal (validation) is handled through MustCompile, so we just need to collect the string

TODO Validate and create regex

The usage part of the flag package is minimal, so we can try just using these bits:

var flagvar int
func init() {
  flag.IntVar(&flagvar, "flagname", 1234, "help message for flagname")
}

Let’s incorporate this in. Notice (r) now uses the value provided by (f). Also notice with (f) that we use flag.Parse. Input is not “parsed” until you run this, so we must make sure to run it after defining the flags!

package main

import (
    "flag";
    "fmt";
    "bufio";
    "os";
    "regexp";
)

var regexFlag string
func main() {

    flag.StringVar(&regexFlag, "e", `.*`, "golang compatible regex expression")
    flag.Parse()

    // TODO Validate and create regex
    re := regexp.MustCompile(`\d\w{2}`)
    re := regexp.MustCompile(regexFlag)
    // TODO Get stdin input
    reader := bufio.NewReader(os.Stdin)
    text, _ := reader.ReadString('\n')
    // TODO Check to see if it matches regex
    m := re.Find([]byte(text))
    // TODO Output matched string
    if m == nil {
        os.Exit(1)
    } else {
        fmt.Print(text)
    }
}

Conclusion

Congratulations! You have a nice minimal functional app with golang. Whenever you feel ready, feel free to see the next app. You know, assuming I’ve gotten around to writing it.

Otherwise, there’s two things you can expand upon to make this app more robust:

If the regex provided is invalid, it will panic. Perhaps use regexp.Compile instead, and check for an error for a more graceful exit? Perhaps just using os.Exit(1)?
There’s no help message. How does someone receiving this figure out how to use it in the first place? To figure this out, consider searching for flag help message golang. flag.PrintDefaults() looks promising. You could trigger this for a number of reasons: Creating a new -h flag, if there is 0 arguments, or if there is an error.