code.sd

November 2023 – 11 March 2025

Migration to Go language

Introduction

Go is compiled, statically typed, concurrent, imperative, with OOP support, high-level programming language. Go has been developed and supported by Google, and version 1 has been released in 2012

Author

My name is Motaz Abdel Azeem, and I graduated at Sudan University of Science and Technology in 1999, and I have been using using Delphi and Object pascal for long time, then Java and now Go language. I have written many books for programming languages: Object Pascal, Java (an Arabic book) and now Go. I’m a founder of Code for Computer software (code.sd)

Main objectives for this book

There are many reasons that why I’m writing this book: First I need to study again Go language in more academic way, and to cover most of language details including that I didn’t used in production. Also to receive feedback from readers and reviewers for our methodology of using Go. Also this book is written to assist to spread - the good language - Go and add more learning books as our contribution to community and as pay-back after our benefit of using Go. Also this book could be a guide and introduction for our new developers whom join Code.sd, to understand why we have choose Go over other languages, and to teach them in fast track to your methodology of using Go.

Book license:

This books is licensed under Creative Commons

How to read this book

If you want to start immediately, it is better to skip the long comparison pages and start from

Install Go chapter. Then later after a while you can return back to comparison sections.

A Brief about us and used technologies:

Code (a Software Firm) is specialized in development in networking, telecom software, APIs, VOIP, and web applications. Previously Java and PHP were been used in development for all our systems and modules, but after 4 to 6 years, problems has arises from both languages in the long run, these problems are:

a). Java problems:

1. Java consumes a lot of resources: CPU and Memory, specially when deploying in Tomcat server. Programs written in Java requires considerable amount of memory in servers – compared with other technologies- for large amount of traffic and after long running time. For development PCs also it consumes a lot of memory to run NetBeans (IDE that is written in Java), also requires powerful CPU. It runs very slow in moderate resources developers PCs.

2. We encounter servers CPU load for Java web applications and web services, also crashes happens in Tomcat which results in stopping all deployed Java web applications and web services on that server.

3. JSP feature of Java encourages developers for bad practice: an anti-pattern of writing logic within presentation and sometimes data access in the same file (.jsp file).

4. Java JVM version differences: There are many servers with different releases of Ubuntu which has different versions of Java, starting from Java 7 to 11, also there are different versions of Tomcat and MySQL, in addition to developer different environment version, this makes incompatibility and errors when deploying with higher version and different version in targeted machines.

5. Oracle is licensing usage of Java SE, in spite of that OpenJDK requires no licensing, but it is not grantee that OpenJDK will be developed, improved, and comparable with new versions of Java.

6. Java run-time in deployment side requires updates on the server for security purposes. Old unpatched Java run-time could have vulnerabilities and expose server to intrusion

7. Java encourages developers to rely on third party frameworks rather than standard libraries. This adds more complexity in learning, development, deployment and troubleshooting

b). PHP problems

1. PHP is a scripting language, so that it requires project source files to be deployed in server side, and in case of in-premises deployment, source code will be exposed to customers and could be changed, and couldn’t be protected against change and copyright.

2. It requires PHP interpreter to be available on servers with specific version and with required packages

3. Upgrading developer PHP version requires upgrading servers version of PHP and all running applications on that server

4. Some developers are modifying code directly in clients servers, which results in different versions of files in servers than developers PCs and source control. This makes conflicts between main trunk version and deployed versions

5. PHP allows developers to write code and HTML in the same file, which breaks single responsibility principle and MVC. This makes code hard to read and to modify

6. Since PHP is not compiled, PHP source code with errors could be deployed or submitted to source control, also change could happen in deployment files that could generate syntax errors.

7. PHP has weak types definition and does not support unit testing and debugging, so that bugs and problems are hard to trace and to fix

8. Scripting languages are fragile in deployment environment, not like compiled and byte code languages that produces solid binaries or byte code that is more immune against change in deployed environments

9. PHP sites are subject to hackers intrusion, they will know that this site is built using PHP and they could find vulnerabilities. Hackers are always targeting PHP open source packages that are often deployed in public servers such as phpMyAdmin, and wordpress.

Go features and advantages

1. Go is a compiled language, and it’s executable binary is self-contained, which means that all packages and dependencies are linked into one single executable that is easy to deploy and run independently of platform version in target machines. Specially when compiled statically (compiling with option CGO_ENABLED=0)

2. It supports cross-compilation: we can produce Linux 64 bit binary from windows OS and vice versa

3. Memory safety and garbage collection compared to similar compiled languages such as C and C++

4. Fast execution for produced binaries

5. Uses low resources: memory and CPU

6. Has simple and readable syntax, hence fast to learn

7. Module support: easy to import packages with specific versions and it’s dependencies

8. Has built-in networking packages, this makes it suitable for networking, Internet, and communication applications.

9. Go-routines: an easy implementation of multi-tasking, light weight for CPU and memory compared to Java heavy threads

10. Simplicity: Go language and tools are very simple, no frameworks are required. Standard language packages and libraries are very rich, this reduces investment in study for the language and it’s basic libraries and packages, also provides new developers fast track to learn and start developing using Go.

11. No hacking target for deployment applications, because there is no platform, or libraries are required to deploy Go applications in servers, so that it is more immune against hacking compared to scripting and run-time based languages.

Go features according to Go site:

https://go.dev

• An open-source programming language supported by Google

• Easy to learn and great for teams

• Built-in concurrency and a robust standard library

• Large ecosystem of partners, communities, and tools

Drawback of Go language

As any programming language, each programming language has been designed to cover specific domain of programming, so that there is no one programming language could fit for all domains. Go language is no exception, and as a newly introduced language it has it’s own problems and lacks in some aspects.

1. It has fewer developers compared to other programming languages that already has dominated development for decades. Our solution to this problem is to hire junior developers and let them learn Go, since Go is simple and easy for learning. Junior developers tend to accept new technologies more than seniors whom tend to resist and keep their old technologies that they already invested their time.

   Some senior developers whom already face problems with their current programming languages could be convinced to migrate to Go.

2. Some customers requires to use specific languages that their internal developers use, in case of delivering software with source code, and they choose other languages that already famous, old, and has larger developers base, such as Java. Part of customers are asking for programming language for curiosity only. Most of customers requires delivery as black-box product

3. Go language is developed and controlled by single company : Google, so that license could be changed in the future, the same as Oracle did for Java. Truly open source languages that has many organizations and companies sharing it’s development is more immune against license change in the future. In other side this also could be considered as strength point, to have one large successful company to produce such programming language and it’s tools the same as commercial competitive products, and this company is one of largest users for Go language in their own systems and services.

4. Module Packages download must be downloaded through Google proxy, even if you host packages in your public server, if they close that proxy server from your country, you couldn’t use Go with external packages

Our success case of using Go

We have started experimenting and re-writing small projects to Go language, then we have started a migration of critical and unstable Java modules. We get an immediate result by achieving stability with newly converted modules to Go, and one of examples is a web service for Asterisk that was unstable Java web service running in Tomcat which always crashes under heavy load. Go version has achieved stability and running for more than 5 years without any crash or stop. Here is the process in Linux server, check process date, it has started since 2018:

ps -eo lstart,cmd | grep goagent
Sun Jul  1 08:35:10 2018 ./goagent

This case becomes a reference point to us and solid result to adopt Go language in our critical modules.

An important thing that worth mentioning on this case, is that some of our unreliable Java web service and background services that has been written after many years developing mostly in Java; has been re-written using Go by junior developers with only two months of experience in Go and we get stable web services.

Successive migrations have been done after that module re-writing, and it always results in a better, stable and cleaner code. But note that we didn’t translate Java code as-it-is in all cases, sometimes we have enhance internal structure for most of migrated code, to implement software engineering principles, but even when no enhancement in code logic and structure is done; we get benefits of performance, stability and low resources usage for all migrated projects and modules, also we could give junior developers a Java or a PHP module to translate it to Go.

Java modules that handles a moderate traffic and didn’t require frequent modifications, and stable, and just working fine, there is no urgent need to migrate them, specially when were written using best practices. The main issue for such projects is that we need to have Java knowledge among team in order to maintain these modules. Our target is to minimize tools, technologies and programming languages, so that if we manage to migrate all or most of our projects and modules to one language and fewer technologies; we could achieve that target and support our running projects with a smaller team.

Go vs Java resources comparison

I have made simple console application timer in both Go and Java, which runs in an infinite loop to display current date and time in console to see how it consumes in resources:

1. GoTimer source:

package main

import (
     "fmt"
      "time"
)

func main() {
   for {
        fmt.Println(time.Now().String())
        time.Sleep(time.Second)
   }
}

2. JavaTimer source:

package javatimer;

import java.util.Date;

public class JavaTimer {
    public static void main(String[] args) throws InterruptedException {
        
        while (true){
            System.out.println(new Date().toString());
            Thread.sleep(1000);
        }
    }
}

Here are memory comparison when running both applications in Linux:

Note that GoTimer consume 1.9 Megabytes, while JavaTimer consumes 18.3 Megabytes:

Using ps_mem python program:

For consumed threads in OS:

using below command in Linux:

ps -o nlwp <pid>

GoTimer spans to 5 process threads while JavaTimer spans to 15 process threads:

To compare it to C, we have written below code in ctimer.c file:

#include<stdio.h>
#include<time.h>

int main() {
    struct timespec waitt = { 1/*seconds*/, 0/*nanoseconds*/};
    while (1) {
      time_t t;   
      time(&t);
      printf("Time: %s", ctime(&t));
      nanosleep(&waitt,NULL);
      fflush(stdout);
    }
    
    return 0;
}

Memory usage of C version of timer is only 109 Kilobytes

Private + Shared = RAM used Program

96.0 KiB + 13.5 KiB = 109.5 KiB a.out

and it is only one process:

ps -o nlwp 44147
NLWP
   1

C is a winner here for resources consumption, but the code is less readable.

Also C is a low level programming language, and in our use case of Go and Java are higher level programming languages that can be used for enterprise software, so that C is not a candidate in our business case.

Threads comparison:

Go has light weight threads called go routines, while Java has more heavy threads.

Here are previous Timer examples modified to have multiple threads, note that Go routine is more simple to write:

1. GoTimer with threads: (20 lines)

package main

import (
    "fmt"
    "time"
)

func displayTime(id int) {
     for {
          fmt.Println(id, time.Now().String())
          time.Sleep(time.Second)
     }
}

func main() {
     fmt.Println("Display Time Go Routine")
     go displayTime(1)
     go displayTime(2)
     select {}
}

2. JavaTimer with threads: (32 lines)

package javatimer;

import java.util.Date;

class JavaTimerThread extends Thread{  
    public int ID;
 
    @Override
    public void run(){  
        while (true){
            System.out.println(ID + ":" + new Date().toString());
            try {
                Thread.sleep(1000);
            } catch (InterruptedException ex) {
                System.err.println("Error: " + ex.toString());
            }
        }
    }
}  

public class JavaTimer {
    public static void main(String[] args) throws InterruptedException {
       
        JavaTimerThread timer = new JavaTimerThread();
        timer.ID = 1;
        timer.start();
        
        JavaTimerThread timer2 = new JavaTimerThread();
        timer2.ID = 2;
        timer2.start();        
    }
}

When we run command to check process internal threads we will notice that GoTimer remains with 5 threads while Java process threads has increased to 17:

Executable size

Another important difference, but this time is on Java side: Java produces small size byte code executable (in kilobytes), while Go compiler produces self-contained large size binary executable, which contains run-time libraries and used packages, all of them will be inside executable that starts with about 2 megabytes in size.

Java is heavily rely on Java virtual machine or Java runtime (JRE), so that byte-code executable are very small and relies on shared JRE between all java applications on that machine Here are our previous comparison projects sizes:

Java

2.5K JavaTimer.jar

Go:

1.9M GoTimer

That was the minimum compiler output size, so that we need to check real projects for multi thousands lines:

C:

17K a.out

C binary has less statically-linked run-time size compared to Go, also C has no garbage collector.

Here is Go 6k lines that uses few packages:

12M CodeAccountingWS

Compared to Java 7k lines that uses few Java class libraries:

5.5M SMSPanel.war

This is not big issue for disk space on which we deploy executable, but it might be an issue when transferring binary files via network/Internet, it could take a time to in case of low speed network connections, so that it is better to compress binaries while copying.

Installing Go

You can download Go compiler from official Go site directly to get the latest versions:

https://go.dev/

After downloading suitable OS version we can follow the “installation instructions” depending on platform we are using.

This package contains Go compiler, debugger, run-time libraries, Go packages, and command line tools that used by developer and used by IDE that we will install later

This is an example of deploying Go package in Linux:

rm -rf /usr/local/go && tar -C /usr/local -xzf go1.20.linux-amd64.tar.gz

or as multiple commands and using sudo:

sudo rm -rf /usr/local/go
sudo tar -C /usr/local -xzf go1.20.linux-amd64.tar.gz

After that we need include Go in Path OS variable, then to check Go compiler has been deployed, and in case of installing new version we need to make sure new version is working now:

go version

Result could be something like this depending on version and target OS:

go version go1.20 linux/amd64

First Go application

After installing Go compiler, we could use any simple text editor to write first Go sample command line project:

First we should create folder by the same project name, for example “first”, then create main.go file inside “first” folder and write below code.

Then we can use gofmt tool to reformat text alignment to Go standards for more readability:

gofmt -w *.go

main.go file contents:

package main

import (
        "fmt"
)

func main() {

     fmt.Println("Hello Go")
}

Then run first application:

go run main.go 

This produces executable binary in a temporary directory and run that binary

Output:

Hello Go

To build go binary output, we can use go build command:

go build main.go

or just go build in case of multiple project files:

go build

After running go build command we will find executable in the same directory of source project code:

-rw-rw-r-- 1 motaz motaz  105 Feb 11 08:31 main.go
-rwxrwxr-x 1 motaz motaz 1.9M Feb 11 08:34 main

Previous sample files represents Linux environment, in Windows we will find main.exe file

We can run executable and distribute it to other machines

To run it in Linux:

./main

And in Windows

main.exe

or

main

As in native compiled languages, we will get a binary executable depending on OS and Platform, and that executable runs only on the same targeted Platform, to check executable in Linux we could use file command tool to check the file compiled target:

file main

The result will be something like this depending on OS:

main: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, Go BuildID=T1xYAYiZPSUBtfa28x_N/Kko4il7vBW6OyEirX06W/9vAOZhoNUt7A7DP7tVTG/Y_meKiaUFP4PphxQag_Q, with debug_info, not stripped

The advantage of Java here -compared to native compiled language- is that Java produced byte code will run on every machine that has compatible version of Java virtual machine, so that only one byte code is required to be produced, and could be run in different platforms in which JVM exists there.

Cross compilation

As Go available on all major platforms, we could compile that source in every platform the same like C and C++, but the advantage here in Go is that there is an easy cross-compilation, which means you can produce compiled binary for any supported platform from your development platform, for example if you are using Linux, you can produce Linux, Windows 64/32, Mac, ARM binaries from your Linux PC.

Here is an example of how to produce Windows 64 bit binary from Linux or any other platform:

GOOS=windows GOARCH=amd64 go build main.go 

and now we will find main.exe file and if we checked it with file main.exe we will get:

file main.exe 
main.exe: PE32+ executable (console) x86-64 (stripped to external PDB), 
for MS Windows

Here another example to build ARM executable that could run on RaspberyPI:

env GOOS=linux GOARCH=arm GOARM=5 go build main.go 

produced executable name will be main, so that your previous Linux binary will be overwritten, and you need to use file command to differentiate between different binaries of Linux, and Unix. Mac binary also has no extension the same like Linux:

file main
main: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, Go BuildID=Fk6aTEOgy7bWmOgkdLVd/KKZ5f1Qe8FS4O-NQHfYY/ce9Vr7NIvgeZAQWdT0HI/FLMnwvzqn4tiG38gW_wB, with debug_info, not stripped

IDE

Go compiler and it’s tools are independent of any development IDEs, it could be used with many IDEs, such as Visual Studio Code, and LiteIDE. These IDEs are using Go compiler and tools to provide syntax check, format, compilation, run, testing, building and debugging.

1. Visual Studio Code

It is one of famous development IDE, and it uses extensions in order to provide support for variety of programming languages.

After installing Visual Code, we can install Go extensions, or start writing Go project, then IDE will suggest extensions to be installed.

First extension will be: Go for Visual Studio Code, then Go runner extension to manage running and stopping Go programs.

2. LiteIDE

LiteIDE is a specific Go IDE, so that it will be ready for Go projects, no extensions or special configuration is required, it will be ready out of the box for Go programs. It is ideal for starting with Go. It provides compilation, building, modules initialization, testing, and cross-compilation.

Also we can easily work in multiple projects in LiteIDE, but it lacks integration with source control, you need to use external tools or command line to import and commit projects in source control.

In our illustration we will use LiteIDE because of previously mentioned reasons, but we will mention command line method for running and building which is used by Visual Code.

First Go sample with LiteIDE

To create new Go project you will find multiple choices, we will choose: Go new command project option.

This creates project in default go/src directory, if you want to create project in another directory you could select Go new command project (Anywhere) option

We named it first-sample, then we will find below ready written source template:

// first-sample project main.go
package main

import (
     "fmt"
)

func main() {
     fmt.Println("Hello World!")
}

If we try to run it using build and run button (BR) or Control+R, we will get below message:

go: go.mod file not found in current directory or any parent directory; see 'go help modules'
Error: process exited with code 1.

LiteIDE enforces to use Go module in spite of that this simple project does not require external package, but we could initialize module for future use of external packages.

Module could be initialized using (M) button menu, then select Go module Init.

For Visual code we don’t have to initialize module unless we need an external package, and in this case we could initialize module using below command line in project directory:

go mod init

go.mod file will be created containing below text:

module first-sample
go 1.20

Then we can run again the project and see the result, also an executable will be produced after building project, either first-sample filename in Linux, Unix, and Mac environment or first-sample.exe in Windows environment

We can add another line to main.go source file to display current date and time by typing:

fmt.Println(time.

Then LiteIDE will suggests to add time package automatically, and code will be:

// first-sample project main.go
package main

import (
     "fmt"
     "time"
)

func main() {
     fmt.Println("Hello World!")
     fmt.Println(time.Now().String())
}

this requires build and run again to get below result:

Hello World!
2023-02-28 08:41:40.236956669 +0200 CAT m=+0.000123066
Success: process exited with code 0.

In below command, we can do temporary build for this program and run it:

go run .

This command will not provide executable in current directory, and if we need to produce executable without running the application we can use below command:

go build

then we can run produced binary using:

./first-sample 

in Linux, Unix, and Mac

or first.sample in Windows

first-sample

We can right-click project in LiteIDE then select Open terminal here

We can assign current time to a variable then use that variable for different purposes such as to display time or to store date and time in a log file, using one of below variable declarations:

1. Using explicit declaration using var keyword:

        var atime time.Time
        atime = time.Now()
        fmt.Println(atime.String())

2. Second method is to declare variable by direct assign of value using (:=) operator:

        atime := time.Now()
        fmt.Println(atime.String())

In this example we will use explicit declaration of atime type, then assign value in the same line:

        var atime time.Time = time.Now()
        fmt.Println(atime.String())

This could also be for any other types, such as integers and strings:

        var area int = 1200
        diameter := 1.2
        unit := "Kilometers"
        fmt.Printf("Area is %d %s\n", area, unit)
        fmt.Printf("Diameter is %f %s\n", diameter, unit)

Note that explicit declaration of variable type makes it more readable to define to readers the type of variable instead of searching function returned value type or guessing assigned value type. IDE also could help showing function returned value by moving mouse pointer to function name as in Visual Code, or pressing Control key and pointing by mouse as in LiteIDE, but sometimes we need to review code using browser when accessing source control, so that explicit delegation will help in this case.

Note that there is no semicolon (;) at the end of lines or statements, which eliminates ; expected compiler errors, such as in other programming languages, but it could be used for multi-statement lines such as:

        diameter := 1.2; unit := "Kilometers"

If we click save on LiteIDE or used gofmt, they will convert it to two lines and removes ;

Arrays and Slices:

Arrays in Go has fixed length of specific type, here is example of array of string:

     var list [3]string
     list[0] = "First"
     list[1] = "Second"
     list[2] = "Third"
     fmt.Println(list)

Output:

[First Second Third]

It could be iterated using for .. range statement:

     for i, item := range list {
          fmt.Println(i, item)
     }

Output:

0 First
1 Second
2 Third

First variable of for .. range (i) will hold index of current iterated item of array while second variable (item) will hold current iterated array element. We can omit index if we need only item by using underscore:

for _, item := range list {

Slices are dynamic, it requires initialization either with zero-length then append it later with items, or initialized with initial size and also it could be appended later:

     var list []string
     list = make([]string, 3)
     list[0] = "First"
     list[1] = "Second"
     list[2] = "Third"
     list = append(list, "Fourth")
     fmt.Println(list)

output:

[First Second Third Fourth]

Maps

Maps in Go is an implementation of hash table, which stores data in a key-value pair, as in below example:

func main() {
     personMap := make(map[string]string)
     personMap["name"] = "Motaz"
     personMap["address"] = "Khartoum,Sudan"
     personMap["title"] = "Developer"
     personMap["dob"] = "1975-11-16"
     fmt.Println("Name = ", personMap["name"])
     fmt.Println(personMap)
     delete(personMap, "dob")
     fmt.Println(personMap)
}

output:

Name = Motaz
map[address:Khartoum,Sudan dob:1975-11-16 name:Motaz title:Developer]
map[address:Khartoum,Sudan name:Motaz title:Developer]

Here we have initialized map of string that their keys are string also, and we could use other types such as map[string]int, that their values are integer and keys as string, or map[int]string: values are string and keys as integer.

We can iterate through map using for .. range statement as in below sample:

     for key, value := range personMap {
            fmt.Println(key, "=", value)
     }

output:

name = Motaz
address = Khartoum,Sudan
title = Developer
dob = 1975-11-16

Passing variables by reference

When calling a function we could call it using constant parameters such as:

        displaySum(2, 5)

or passing parameters using variable by copy:

        a:= 5
        b:= 6
        displaySum(a, b)

Third option is to pass parameter by reference, in which we need to let function to be able to change variable value, for example swap function:

In this case we have to define parameters as pointers:

func swapNumbers(a *int, b *int) {
       
   *a, *b = *b, *a
   return
}

When calling that function we have to pass address instead of values (&a, &b):

     swapNumbers(&a, &b)

Here is another example using string type: a function to change spaces to underscore in names:

func toUnderscore(name *string) {

     *name = strings.ReplaceAll(*name, " ", "_")
     return
}

*name = means the contents of pointer (name)

We can call it by passing variable address (&x):

      name := "go language"
      toUnderscore(&name)
      fmt.Println(name)

We can make a general swap function using interface{} type or it’s new alias type (any) :

func swap(a, b *any) {
        *a, *b = *b, *a
        return
}

We can call it as:

        var a, b any
        a = "a"
        b = "b"
        swap(&a, &b)
        fmt.Println(a, b)

Generics

Generics has been added to Go in version 1.18, instead of using dynamic empty interface{} or it’s any alias type that is checking type at run-time and could generate panic error which could close application, a new method has been introduced, in this method compiler could check types at compile time and prevents errors. Here is an example of add function that could be used to add integers, float or concatenate strings:

func add[T int | float64 | string](a T, b T) (result T) {
        return a + b
}

This generic add function could be called as in below example:

        fmt.Println(add(2, 3))
        fmt.Println(add(1.2, 2.1))
        fmt.Println(add("Hello", " World"))

Using this generic method is more safer and faster than interface{}, the compiler will generate multiple instances of add function for each type. This represents static and strong typing discipline in Go language

Writing to text file

Here is an example of how to write string in text file, if file does not exist; writeToFile function will create it, and if file already exists it will append to it:

package main

import (
     "fmt"
     "os"
     "time"
)

func main() {
     fmt.Println("Writing to text file")
     writeToFile("text.txt", "Time in server is: "+time.Now().String())
}

func writeToFile(filename string, text string) (err error) {

     var file *os.File
     file, err = os.OpenFile(filename, 
                     os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
     if err == nil {
          defer file.Close()
          file.WriteString(text + "\n")
     }
     return
}

We have used point to type os.File in “os” package, so that we have added it to import section.

In writeToFile function we have called os.OpenFile with create and append options, if opening/creating file has succeeded and err is nil, it will write text to file, but note that we have used defer keyword statement in front of closing file (f.Close()). Defer do two things: 1: delays execution of given statement to end of current block or function, and 2: ensures execution of that statement in all cases, such as error or panic, except if explicit Exit command is called.

File mode parameter that contains 0644 is a Unix permission type giving full read-write permission to owner, and read-only for groups and others.

Reading text file

In this example we have written two ways to read from text file: first method (readLinesFromFile) is to read line by line, and second method (readEntireFile) that reads all contents at once :

package main

import (
     "bufio"
     "fmt"
     "io/ioutil"
     "os"
)

func main() {
     filename := "main.go"
     fmt.Println("Reading from text file: ", filename)
     //readLinesFromFile(filename)
     readEntireFile(filename)
}

func readLinesFromFile(filename string) (err error) {
      
     var file *os.File
     file, err = os.OpenFile(filename, os.O_RDONLY, 0644)
     if err == nil {
             defer file.Close()
             scanner := bufio.NewScanner(file)
             for scanner.Scan() {
                     line := scanner.Text()
                     fmt.Println(line)
             }
     } else {
             fmt.Println("Error: ", err.Error())
     }
     return
}

func readEntireFile(filename string) (err error) {
      
     var file *os.File
     file, err = os.OpenFile(filename, os.O_RDONLY, 0644)
     if err == nil {
             defer file.Close()
             var contents []byte
             contents, err = ioutil.ReadAll(file)
             if err != nil {
                     fmt.Println("Error reding file : ", err.Error())
             }
             fmt.Print(string(contents))
     } else {
             fmt.Println("Error: ", err.Error())
     }
     return
}

I think the code is clear and needs no more clarifications, it reads all file contents in an array of byte, then we could typecast if to string to show it in console.

We can read filename from command line arguments by modifying main function to:

func main() {
      
     if len(os.Args) != 2 {
           fmt.Println("Usage:")
           fmt.Println("readtextfile <filename>")

     } else {
           filename := os.Args[1]
           fmt.Println("Reading from text file: ", filename)
           //readLinesFromFile(filename)
           readEntireFile(filename)
     }
}

Note that first argument (os.Args[0]) is program executable name itself with it’s path

We could call it from command line in Linux and Max as:

./readtextfile main.go  

And in Windows:

readtextfile main.go 

Date and Time format

As we have already displayed time using time.Now().String() which always display date and time in default format, eg:

2025-03-09 08:27:53.940826609 +0200 CAT m=+0.000124256

If we need to change presentation format of date or time, we could use Format function, for example to display short date time format:

        fmt.Println(time.Now().Format(time.DateTime))

2025-03-09 08:38:40

And to display date only and time only :

        fmt.Println(time.Now().Format(time.DateOnly))
        fmt.Println(time.Now().Format(time.TimeOnly))

Output:
2025-03-09
08:29:54

For more custom date and time format we need to use format for search day and time element, for example if we want to display Month in specific position, Year as short or long, Hour as 24 format or 12 format, and Day of week. In this case Go have provided strange method for date and time formatting. In many programming language they use (y) for year, (m) or (n) for month, but for Go they use (1) symbol for month, and (2) for day and (2006) for year example:

        fmt.Println(time.Now().Format("2-1-2006"))

This will display current date for example March 9’th 2025:

9-3-2025

For time, they choose (3) for hours, (4) for minutes, and (5) for seconds:

        fmt.Println(time.Now().Format("3:4:5"))

Current time example output:

8:49:56

For month name they use (January) and for Week day name they use (Monday):

        fmt.Println(time.Now().Format("Monday 2 January 2006"))

Output for 9’th day of March 2025 :

Sunday 9 March 2025

And here an example of displaying current date and time including short week day (Mon), and short month (Jan) name, and time zone (MST):

        fmt.Println(time.Now().Format("Mon, 02 Jan 2006 15:04:05 MST"))

This will display current date and time :

Sun, 09 Mar 2025 08:56:13 CAT

Note that (3) is used for hour for 12 hours format, and (15) is used for 24 hours format.
For 12 hours format we need to display AM/PM indicator, we use (PM) symbol to display it

        fmt.Println(time.Now().Format("03:05 PM"))

Output sample:

08:21 AM

If we want to remember date and time format symbol numbers, we have to arrange them sequentially to understand why January has been chosen, and why 2’d day of month, and why 2006 year, and 3 has been chosen to hours and 4 for minutes and 5 for seconds:

1, 2, 3, 4, 5, 2006:

fmt.Println(time.Now().Format("Month 1, Day 2, Hour 3, Minute 4, Second 5, Year 2006"))

Output:

Month 3, Day 9, Hour 8, Minute 3, Second 46, Year 2025

To receive date and time as input from user, we have to choose format also, for example if we asked user to enter date format as dd/mm/yyyy, then we have to use this format and parsing function:

func main() {
        fmt.Print("Please input your birth date in dd/mm/yyyy format:")
        var birthDateStr string
        fmt.Scanln(&birthDateStr)
        fmt.Println()
        birthDate, err := time.Parse("2/1/2006", birthDateStr)
        if err != nil {
                fmt.Println("Error parsing date: ", err.Error())
        } else {
                fmt.Println("Your birth date is: ", 
                   birthDate.Format(time.DateOnly))
        }
}

If an invalid date format has been entered or out of scope numbers; an error in parsing will be returned

Go routines

Go routines is a way of implementing parallel running for code and using multi-core CPU. Go routine is lighter than Java thread, here is sample using of Go routines:

package main

import (
     "fmt"
     "time"
)

func main() {
     
     go displayTime("1")
     go displayTime("2")
     go displayTime("3")
     for {
     }
}

func displayTime(id string) {

    for i := 0; i < 3; i++ {
        fmt.Printf("Routine : %s time in server is: %s\n", id, 
            time.Now().String()[:19])
        time.Sleep(time.Second)
    }
}

Output:

Routine : 1 time in server is: 2023-04-11 08:27:34
Routine : 2 time in server is: 2023-04-11 08:27:34
Routine : 3 time in server is: 2023-04-11 08:27:34
Routine : 1 time in server is: 2023-04-11 08:27:35
Routine : 2 time in server is: 2023-04-11 08:27:35
Routine : 3 time in server is: 2023-04-11 08:27:35
Routine : 3 time in server is: 2023-04-11 08:27:36
Routine : 1 time in server is: 2023-04-11 08:27:36
Routine : 2 time in server is: 2023-04-11 08:27:36

We have only added go keyword to run displayTime function in background as that simple.

Three routines will run simultaneously

Infinite for loop is important to keep main program wait that execution, otherwise application will exit before executing any routine. We could also add timer instead of infinite loop, because using infinite loop will not exist unless it has been closed using Ctrl-C or ending it from tasks or processes manager

     go displayTime("1")
     go displayTime("2")
     go displayTime("3")
     time.Sleep(time.Second * 3)

Wait Group

If we have Go routines or multiple Go routines that we want to run in parallel and waiting their results before proceed to next statements, we can use WaitGroup in sync package.

We have modified previous go routine sample that has three times for loop to use WaitGroup instead of infinite for loop at end of program:

package main

import (
   "fmt"
   "sync"
   "time"
)

func main() {
     var wg sync.WaitGroup
     wg.Add(3)
     go displayTime(&wg, "1")
     go displayTime(&wg, "2")
     go displayTime(&wg, "3")
     wg.Wait()
     fmt.Println("All routines has finished")
}

func displayTime(wg *sync.WaitGroup, id string) {

   for i := 0; i < 3; i++ {
         fmt.Printf("Routine : %s time in server is: %s\n", id, 
                time.Now().String()[:19])
         time.Sleep(time.Second)
   }
   wg.Done()
}

In this sample program we have declared wg variable as sync.WaitGroup type, and have initialized it for three routines using Add method ( wg.Add(3) ) and we will wait until all these three routines to finish. We have passed wg variable by reference ( &wg ) to displayTime function to call Done() method of wait group.

At main function after calling go routines we have called wg.Wait() to wait all routines to finish before proceed with next statements of program, and we could get result back for another process, such as fetching data from web in parallel then do process for that data

Output:

Routine : 3 time in server is: 2023-04-11 09:13:20
Routine : 1 time in server is: 2023-04-11 09:13:20
Routine : 2 time in server is: 2023-04-11 09:13:20
Routine : 2 time in server is: 2023-04-11 09:13:21
Routine : 1 time in server is: 2023-04-11 09:13:21
Routine : 3 time in server is: 2023-04-11 09:13:21
Routine : 2 time in server is: 2023-04-11 09:13:22
Routine : 3 time in server is: 2023-04-11 09:13:22
Routine : 1 time in server is: 2023-04-11 09:13:22
All routines has finished

Note that it is better to call Done() at the top of function with defer keyword to ensure calling it at any cases, and not let Wait() wait forever in case of crash inside function before reaching Done() method:

func displayTime(wg *sync.WaitGroup, id string) {
      
   defer wg.Done()
   for i := 0; i < 3; i++ {
       fmt.Printf("Routine : %s time in server is: %s\n", id, 
          time.Now().String()[:19])
       time.Sleep(time.Second)
   }
}

Mutex

One of important thing in Go routines and threads is shared resources access, for example if we look back to our write to text file sample and tried to write on that file using multiple go routines, a race condition will occur, which means trying to access resource exclusively, in this case to write on file, such race condition could corrupt file or crashes the program. To prevent race condition, we have to synchronize writing to that file, by enabling only one thread/go routine to write to that file and lock the resource while writing, then release that lock after finish writing, while enforce other routines to wait first routine until finish.

Mutex (Mutual Exclusion) in Go enables lock and unlock for any resource, here write to file sample program rewritten using go routines and mutex :

package main

import (
     "fmt"
     "os"
     "sync"
     "time"
)

var mutex *sync.Mutex
var counter int = 0

func main() {

     mutex = &sync.Mutex{}
     fmt.Println("Writing to text file")
     for i := 0; i < 10; i++ {
             go writeToFile("text.txt", 
     fmt.Sprintf("routine # %d: Time in server is: %s",
                     i+1, time.Now().String()))
     }
     time.Sleep(time.Second * 5)
     mutex.Lock()
     defer mutex.Unlock()
      fmt.Println("Written counter: ", counter)
}

func writeToFile(filename string, text string) (err error) {
        mutex.Lock()
        defer mutex.Unlock()
        counter++
        var file *os.File
        file, err = os.OpenFile(filename, os.O_APPEND|os.O_CREATE|
           os.O_WRONLY, 0644)
        if err == nil {
                defer file.Close()
                file.WriteString(text + "\n")
        }
        return
}

Now we have two shared resources: external text file, and counter global variable, which they should be accessed in synchronized method for go routines.

We have defined mutex as pointer of type sync.Mutex globally in program and initialized it at main function

We used Lock() method of mutex before access shared resource, and release lock using Unlock() method after writing/reading from that resource:

        mutex.Lock()
        fmt.Println("Written counter: ", counter)
        mutex.Unlock()

We can test our programs against race condition using below command line:

go run -race main.go 

or

go build --race main.go

then run it as main executable normally

In our case we will get normal execution and result:

Writing to text file
Written counter:  10

If we commented out mutex.Lock() and mutext.Unlock() in writeToFile method

        //mutex.Lock()
        //defer mutex.Unlock()

and build the application then run go run -race main.go again, we will get race condition:

==================
WARNING: DATA RACE
Read at 0x0000005f8780 by main goroutine:
  main.main()
      /home/motaz/………/samples/textfiles/main.go:24 +0x266

Previous write at 0x0000005f8780 by goroutine 15:
  main.writeToFile()
      /home/motaz/………/samples/textfiles/main.go:31 +0x84
  main.main.func1()
      /home/motaz/………/samples/textfiles/main.go:19 +0x53

Goroutine 15 (finished) created at:
  main.main()
      /home/motaz/……/samples/textfiles/main.go:19 +0xe4
==================
Written counter:  10
Found 3 data race(s)
exit status 66

Race condition happens on below lines:

        counter++

and

        fmt.Println("Written counter: ", counter)

Note that race condition isn’t detected in writing to text file, only on global variable (counter) writing and reading while others are writing. This might mean that writing to a file in Go is thread-safe, but it is not guaranteed in all cases and all platforms, we are here to demonstrate the need and mechanism of using Mutex

HTTP Web Sample

Go supports HTTP web applications (backend) easily in standard networking packages. HTTP application could be either web application or web service, here is our sample application for web app:

We have created a new project as command line project with name first-web, then we have initialized module in LiteIDE.

When using Visual Code or other editor we will not need to initialize module, because we are using only standard packages in this sample:

// first-web project main.go
package main

import (
     "fmt"
     "net/http"
     "time"
)

func main() {
     http.HandleFunc("/", index)
     fmt.Println("Listening on \nhttp://localhost:9090")
     err := http.ListenAndServe(":9090", nil)
     if err != nil {
          fmt.Println("Error while listening to port 9090: " + err.Error())
     }
}

func index(w http.ResponseWriter, r *http.Request) {

     w.Header().Set("content-type", "text/html")
     fmt.Fprintf(w, "<h2>Go Web Page</h2>")
     fmt.Fprintf(w, "<font color=blue>Main index page</font>")
     fmt.Fprintf(w, "<br/>Today is %s", time.Now().String())
}

Main important function here is http.ListenAndServe(":9090", nil), which will work as an embedded HTTP web server, listening on port (9090 in this example), and this lets program holds and keep working, and makes it a multi-threaded application server listening and serving all requests on that port.

If port is used by other process, or no permission to use (ports under 1024 requires admin permission), ListenAndServe will exit and returns error in err object and our command application will closes in this case.

Second important method is the handler: index() method that we have written to receive HTTP client requests (such as browser request through URL, curl or any HTTP client)

We have linked HTTP path request of (/) to index() handler using HandleFunc method:

     http.HandleFunc("/", index)

We could add another path handler such as /hello after writing hello() method handler and link it using HandleFunc:

     http.HandleFunc("/hello", hello)

func hello(w http.ResponseWriter, r *http.Request) {
     w.Header().Set("content-type", "text/html")
     fmt.Fprintf(w, "<h2>Hello Go</h2>")
}

This is a simple method for writing web pages in Go, but in real development we use templates that we will talk about it later.

Web services

Web service is using the same HTTP package and similar to web page in Go, except it has no user interface and is not targeting browsers, so that there is no HTML here, instead; it targets calling from other applications, but it is still can be called for - testing purposes - from browsers, command line tools, and browser web services plugins.

Here is a simple web service that returns server’s date and time in JSON format:

// first-service project main.go
package main

import (
     "encoding/json"
     "fmt"
     "net/http"
     "time"
)

func main() {
     http.HandleFunc("/time", gettime)
     fmt.Println("Listening on \nhttp://localhost:9090")
     err := http.ListenAndServe(":9090", nil)
     if err != nil {
        fmt.Println("Error while listening to port 9090: " + err.Error())
     }
}

type TimeResponseType struct {
     Time string
}

func gettime(w http.ResponseWriter, r *http.Request) {

     w.Header().Set("content-type", "application/json")
     var atime TimeResponseType
     atime.Time = time.Now().String()
     res, _ := json.Marshal(atime)
     w.Write(res)
}

Note that the first difference here is content-type type, which is application/json in web services instead of text/html in web pages. Also we have used struct type to return JSON response object:

type TimeResponseType struct {
     Time string
}

We call the web service using below request URL:

http://localhost:9090/time

We will get below JSON response:

{"Time":"2023-03-10 09:32:39.072559162 +0200 CAT m=+4.697942371"}

We can add any other type as fields on this struct object (TimeResponseType), but we have to capitalize first letter of field names to make it public, if we write first letter of struct field name in small letter it will be private and will not be displayed in JSON response.

If we want to display it in JSON format as lower-case or in another name we can add JSON formatted as below:

type TimeResponseType struct {
     Time string `json:"time"`
}

Another example :

   Time string `json:"server-time"`

This will make field naming in JSON independent of Go field naming

Simple API sample

We have write another API sample that has three methods:

1. addorder: to send order information

2. removeorder: to remove already sent order

3. getorders: to list current exist orders

In this sample we use text file to store orders to make it simple and to focus for web services in Go without database connection complications.

Sample has been uploaded into Github repository:

https://github.com/motaz/go-api-sample

Here is main file of go-api-sample project:

package main

import (
     "fmt"
     "net/http"
)

func main() {
        http.HandleFunc("/addorder", AddOrder)
        http.HandleFunc("/removeorder", RemoveOrder)
        http.HandleFunc("/getorders", GetOrders)
        http.HandleFunc("/", About)
        port := "10022"
        fmt.Println("http://localhost:" + port)
        err := http.ListenAndServe(":"+port, nil)
        if err != nil {
                fmt.Println("Error while listening: ", err.Error())
        }
}

Here is request sample for /addorder method:

{"Phone":"123456789","Name":"Motaz","Address":"Khartoum"}

Here is parsing part of order:

type ordertype struct {
        Phone   string
        Name    string
        Address string
}

func AddOrder(w http.ResponseWriter, r *http.Request) {
        w.Header().Add("content-type", "application/json")
        requestBytes, err := io.ReadAll(r.Body)
        if err != nil {
                fmt.Fprintf(w, err.Error())
        } else {
                var request ordertype
                err := json.Unmarshal(requestBytes, &request)
…

You can see or download complete project from Github:

https://github.com/motaz/go-api-sample/

Dynamic and Static linking

As we have mentioned in Go Advantages paragraph: Go produces self-contained executable, which means it does not require external dependencies such as run-time or libraries and packages needs to be available in target platforms in order to run applications. Instead this self-contained executable requires only OS kernel to run, but there are some exceptions when requiring external libraries to load them at run-time, this ensures flexibility over self-containing.

There are two types of linking executable during compilation and linking:

First type is Static Linking which means all required packages and run-time will be included inside executable.

Second type is Dynamic Linking, which means they use cgo compiler to link with external C libraries, if these libraries aren’t exist in targe machine, executable will fail to run.

By default Go linker tend to produce Static Linking executable if there is no need to call external C library, but if any package requires such library, the linker will produce Dynamic Linking executable.

To know if specific application produced executable is Dynamic or Static we use file in Linux environment, e.g.:

Simple program: textfiles:

$ file textfiles 

textfiles: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, Go BuildID=6YZ87jE8t_9-qktcWy5B/twyGz9uML0gFmcUjUlrO/OVlj6TwjC4XIgViApurq/O0G8a0PQkQywcE647rdA, with debug_info, not stripped

In the result it says explicitly (Statically Linked)

Second sample for go-api-sample application which uses http that uses external library:

$ file go-api-sample 

go-api-sample: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, Go BuildID=HM6J8MMm_B4_UjZ-wcas/OBkmOXG0MqTcDV6dpOr9/UjkKANT6eQElvvExaDgM/M6DuF-_CXrw-GEsoJMIP, with debug_info, not stripped

We found that this executable is Dynamically linked, and it uses /lib64/ld-linux-x86-64.so.2 library in Linux which is repressible of loading another libraries at run-time.

We can force this application to be produced as Static instead of Dynamic linking using CGO_ENABLED flag, is to but 0 to it, to prevent cgo compilation and linking:

$ CGO_ENABLED=0 go build
$ file go-api-sample 

go-api-sample: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, Go BuildID=m0PGRNBunV5SmICByM5g/kBwhyg0CKlMDRMol7C9d/d2mc8EVlxTjxVKSgDZv2/-uW5EwJaKsAXEMrlMIzq, with debug_info, not stripped

And we can see that obviously that this time executable file has been produced as Statically Linked, this makes it more self-contained and could prevent the problem of not found library error when deploying in target machine.

Another way to determine if executable is statically linked or dynamically linked is by using ldd Linux command:

In case of dynamic linking:

$ ldd go-api-sample 
        linux-vdso.so.1 (0x00007ffefb9e7000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd85f725000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fd85f91c000)

It shows required libraries as above.

In case of static linking:

$ ldd go-api-sample 
        not a dynamic executable

Statically linked executable seems to be better when possible for more self-contained and less rely on target platforms dependencies, but we need to use Dynamic linking in some cases such as when using C code and calling C libraries inside Go application and packages, also when using plugins mechanism to reduce re-compilation in case of changing external library or plugin.

Go HTML templates

Go has templates processors, which enables web developers to isolate HTML design from Go code to achieve single responsibility principle and MVC in web development.

Not like PHP and JSP which allows developers to write code inside presentation (HTML) files, Go language has a simple HTML tags notation that could be used inside HTML to do presentation functions, such as displaying variables, iterating through slice to fill a table, and limited if conditions. This notation is a bridge between Go code which represents controller and presentation HTML pages. This notation is called Go template language.

We have created for templates sample a new project called (webpage) and created new sub-folder inside project folder called (templates) in which we can put HTML pages, and we have created new file called index.html with below contents:

<html>
<head>
<title>Go Template</title>
<body>
  <h3>Go HTML Template web application</h3>
  <p>
    Time in server is: <b> {{.Time}} </b>
    <br/>
    Your IP address is: <b> {{.IP}} </b>
  </p>
</body>
</html>

Note that {{.Time}} and {{.IP}} tags will be replaced later by values passed from Go code.

Here main.go file:

package main

import (
     "fmt"
     "html/template"
     "net/http"
     "strings"
     "time"
)

var mytemplate *template.Template

func main() {
     mytemplate = template.Must(template.ParseGlob("templates/*"))
     fmt.Println("http://localhost:9090")
     http.HandleFunc("/", index)
     err := http.ListenAndServe(":9090", nil)
     if err != nil {
           fmt.Println("Error while listening to port 9090")
     }
}

type TemplateData struct {
     IP   string
     Time string
}

func index(w http.ResponseWriter, r *http.Request) {
     var data TemplateData
     data.IP = r.RemoteAddr
     if strings.Contains(data.IP, ":") {
          data.IP = data.IP[:strings.Index(data.IP, ":")]
     }
     data.Time = time.Now().String()[:19]
     mytemplate.Execute(w, data)
}

We have defined mytemplate global variable as point to template.Template type, then initialized it to load all HTML files in (templates) directory:

      mytemplate = template.Must(template.ParseGlob("templates/*"))

We have used our own struct type (TemplateData) to encapsulate all our required variables and data to be displayed in HTML page

Then we called it inside index handler to display HTML contents and passing required variables/tags to it:

      mytemplate.Execute(w, data)

r.RemoteAddr returns client IP address with port number such as: 27.0.0.1:43060 and we want to remove the port using strings package:

      data.IP = r.RemoteAddr
      if strings.Contains(data.IP, ":") {
              data.IP = data.IP[:strings.Index(data.IP, ":")]
      }

We can copy part of string using [:] for instance: copy first two characters:

        name := "Ahmed"
        name = name[:2]

and last two characters as:

        name = name[len(name)-2:]

and two letters from the middle as:

        name = name[2:4]

here is the output page in browser:

http://localhost:9090/

Go HTML Template web application

Time in server is: 2023-04-10 07:26:24
Your IP address is: 127.0.0.1

Static contents in Web applications

We can add front-end static contents to Go web application such as: images, css, and java script in directories and make that directory accessible to clients, from our HTTP web server using below as in our below sample.

Below sample is part of GoCat project (which is similar to Tomcat), this web application is used for Go web applications and Web services to deploy and mange that applications, complete source exist in this repository:

https://github.com/motaz/gocat

In main.go file before calling http.Listen..

     fs := SetCacheHeader(http.FileServer(http.Dir("static")))
        http.Handle("/gocat/static/", http.StripPrefix("/gocat/static/", fs))

We have a directory called static contains css sub-folder and images sub-folder:

We can access static contents from HTML template as in below HTML header tags:

<link rel="shortcut icon" href="/gocat/static/images/golang.png" />            
<link href="/gocat/static/css/style.css" rel="stylesheet" type="text/css">

Deploying Web apps and Web services

In production it is not practical to have many Go web applications and web services with different ports and enable/open that ports in Firewall. Support that we have 3 apps in the same server listening on ports 9090, 9091, and 9092, and more apps could be added later that requires opening new ports if deployment server is behind a firewall, instead we could use web server such as Apache or Nginx to host our applications. Nginx is more easy and ready to work as reverse proxy, here is sample of entry in sites-enabled/default file for one of our Go web apps or web services listening on port 9090:

        location /timer/ {
                proxy_pass http://localhost:9090/;
        }

We can access it using default port 80 for HTTP or port 443 for HTTPS depending on which section we have defined our reverse proxy entry, we can access it from browser using below URL:

http://localhost/timer

We have to use relative path in our web applications, or we can define a constant path prefix such as:

        http.HandleFunc("/goweb", index)
        http.HandleFunc("/goweb/time", timer)
        http.HandleFunc("/goweb/login", login)
        http.HandleFunc("/goweb/logout", logout)
        http.HandleFunc("/goweb/users", users)
        err := http.ListenAndServe(":9090", nil)

In this case we could define our reverse proxy in Nginx as:

        location /goweb/ {
                proxy_pass http://localhost:9090/goweb/;
        }

We can access it in browser or any client side as:

http://localhost/goweb

http://server-ip/goweb

Nginx could also be used as load balancer, we could host Go web applications and web services in multiple machines and configure load balancer and configure reverse proxy for them

HTTP client

We can write HTTP client using http package that we have already used in our previous web service sample and web app examples, also it can be used to call that web service.

In this sample we will call IP location web service http://ip2c.org giving it public IP address to return country code and name that given IP belongs to:

// http-client project main.go
package main

import (
    "fmt"
    "io/ioutil"
    "net/http"
)

func main() {
     ip := "41.209.90.89"
     result := callHTTP("https://ip2c.org/" + ip)
     fmt.Println(result)
}

func callHTTP(url string) (result string) {
      
   resp, err := http.Get(url)
   if err != nil {
        fmt.Println("Error: ", err.Error())
   } else { 
       body, err := ioutil.ReadAll(resp.Body)
       if err != nil {
           fmt.Println("Error in reading result: ", err.Error())
        } else {
           resp.Body.Close()
        }
        result = string(body)
   }
   return
}

Note that we have write new function (callHTTP) that accepts URL, sends HTTP Get request, then returns response in result parameter.

This is the sample result of running above example:

1;SD;SDN;Sudan

We can add more features to callHTTP function, to include timeout, request content-type, to return content-type, and HTTP status. Here is modified code:

package main

import (
      "fmt"
      "io/ioutil"
      "net/http"
      "time"
)

func main() {
     ip := "41.209.90.89"
     status, result := callHTTP("https://ip2c.org/" + ip)
     fmt.Println("Response:")

     fmt.Println(status)
     fmt.Println(result)
}

func callHTTP(url string) (status, result string) {

     timeout := time.Duration(10 * time.Second)
     client := http.Client{
             Timeout: timeout,
     }
     req, _ := http.NewRequest("GET", url, nil)
     req.Header.Set("Content-Type", "text/plain")
     resp, err := client.Do(req)
     if err != nil {
           fmt.Println("Error: ", err.Error())
     } else {
           status = resp.Status
           fmt.Println("Header Items")
           for key, item := range resp.Header {
                  fmt.Println(key, ": ", item[0])
           }

           body, err := ioutil.ReadAll(resp.Body)
           if err != nil {
                  fmt.Println("Error in reading result: ", err.Error())
           } else {
                  resp.Body.Close()
           }
           result = string(body)
    }
    return
}

Here is execution result:

Header Items
Content-Type : text/html; charset=UTF-8
Access-Control-Allow-Origin : *
Server : nginx
Date : Tue, 04 Apr 2023 04:31:18 GMT
Response:
200 OK
1;SD;SDN;Sudan (the)

We have set timeout of calling that web service to 10 seconds, then we have set Content-Type header value to text/plain, but this is used with POST method to indicate content type that been sent to web app/service.

Note that our function now is returning two values, and this is a remarkable features in Go language; is to return multiple values in functions. We can ignore any of returning values using underscore character ( _ ), for example:

    _, result := callHTTP("https://ip2c.org/" + ip)

We couldn’t assign value to a variable without using it in Go, so we have to ignore returning values that we will not use when calling functions.

Response header is define as map of string :

type Header map[string][]string

and their values are array of string:

We do iteration in the map using for .. range statement:

   for key, item := range resp.Header {
          fmt.Println(key, ": ", item[0])
   }

GoCat manager

We have developed GoCat manager which is similar to Java’s Apache Tomcat, which allows deployment and managing Go web services and web apps, but the difference is that GoCat is not a web container and not a web server, we still need Nginx as we have described previously to access Go web services using standard HTTP port 80 or HTTPS port 443, and to manage load balancing.

GoCat helps to upload, update and monitor Go services and web apps easily.

For GoCat download please visit below page:

https://github.com/motaz/GoCat

MySQL connection and third party packages

In this sample we will connect to MySQL database, and this requires an import of additional package which is not part of standard Go libraries and packages, so that we need to initialize Go module in this sample project to let the additional package be installed from Internet.

Note that this project will be found in sample projects in this book page, with name: mysql-sample

We have used two additional packages in this project:

1. github.com/go-sql-driver/mysql

which is mysql connection library for Go

2. github.com/motaz/codeutils

codeutils is a package that contains functions to read from configuration file (.ini) and to write into log file

we put them in module, and this is source of go.mod file after initializing Go module using LiteIDE Module menu button:

Go module Init

or using command line:

go mod init

We will get below text in go.mod file:

module mysql-sample

go 1.20

After that we can import that packages from their location using below command, in LiteIDE we can click on Module menu button then select

Go module Tidy

or using command line when using other IDEs:

go mod tidy 

Required packages will be imported from net and go.mod file will be updated to:

module mysql-sample

go 1.20

require (
      github.com/go-sql-driver/mysql v1.7.1
      github.com/motaz/codeutils v1.0.20
)

Imported packages will be cached locally in Go package folders (GoPath/pkg/mod) according to their version.

Version is an important tag of Go package, if you write application and depend on specific version of any package, other developers whom need to compile or modify your source will get the exact version of required package when run go mod tidy command, this ensures that your application behavior will not change if new version of package has been released.

Here are mysql-sample project folder files:

We have divided code into three Go source files to implement single responsibility principle:

1. main.go which contains start of project code. Code here is very few because main responsibility of main.go code is to start project and it show be minimum to possible for easy understanding of project.

2. dataaccess.go: this source file contains mysql database access function, like connection, reading from tables, and updating data.

3. controller.go: instead of calling database access functions directly from main.go file, it is better to have middleware layer (business layer/controller layer) to have logical and abstract functions, for example if we use Login method, in this layer we could check username validity, and do hashing for password and make sure that this user is not restricted, then call lower level functions in access layer such as getUserInfo. Actual authentication and authorization logic will happen in controller layer not in data access layer, also external authentication could be used here by calling remote web service or LDAP server, or in another access layer file but not in MySQL access layer file.

Other files are:

1. go.mod: the file that contains required packages and project name. This file is important for project and should be included in source control

   module mysql-sample

go 1.20
require (
        github.com/go-sql-driver/mysql v1.7.1
        github.com/motaz/codeutils v1.0.20
)

2. go.sum: automatically generated file after using tidy tool, this file contains information about locally cached versions of required packages. If cache already exists, then no need to import from remote repository. This file shouldn’t be included in source control, because it will be generated automatically after caching packages in each developer machine.

   github.com/go-sql-driver/mysql v1.7.1 h1:lUIinVbN1DY0xBg0eMOzmmtGoHwWBbvnWubQUrtU8EI=

github.com/go-sql-driver/mysql v1.7.1/go.mod h1:OXbVy3sEdcQ2Doequ6Z5BW6fXNQTmx+9S1MCJN5yJMI=
github.com/motaz/codeutils v1.0.20 h1:fQ6wdkGwUQwsqOyoGocreF9leDHhQzNlPRPtMNV3LL4=
github.com/motaz/codeutils v1.0.20/go.mod h1:3Nwz4asoggDXCVd0Z+5Wf4BPKa2lXZt3tjniSlILN40=

As a hacking method if there is no Internet while creating new project, this file (go.sum) could be copied manually to new project folder in case of using the same packages, but packages should be already importer before for previous projects, and tidy will not be required because it requires Internet access to download from remote repository

3. config.ini: this is a configuration file and it is not part of source. We used codeutils.GetConfigValue function to read user name and password of database instead of fixing them inside code, to allow portability when deploying in different MySQL server. This file shouldn’t be included in source control, each developer should have different local file to represents local MySQL configuration.

   This is a sample of configuration data inside config.ini

   dbuser=mysqluser

        dbpassword=mysecretpass

3. dbscript.sql: this contains database schema which is used by this project and it is part of project source and should be included in source control, note that it is not part of Go project, but it is required to create database before running application.

CREATE TABLE `sample`.`users` (
  `id` INT NOT NULL AUTO_INCREMENT,
  `username` VARCHAR(45) NULL,
  PRIMARY KEY (`id`));

Last file mysql-sample - which is not part of project source code and shouldn’t be included in source control - the output of complication which is binary executable for Unix like systems, and in Windows it will be mysql-sample.exe

Back to Go source files, we will start with lower level files, which means are close to OS, resources, Databases and far from user interface.

1. dataaccess.go:

package main

import (
      "database/sql"
      "fmt"
      "strings"
      _ "github.com/go-sql-driver/mysql"
      "github.com/motaz/codeutils"
)

func GetConfigurationParameter(param, defaultValue string) string {

     value := codeutils.GetConfigValue("config.ini", param)
     if value == "" {
        value = defaultValue
     }
     return value
}

func WriteLog(event string) {

     codeutils.WriteToLog(event, "log")
}

func SQLConnection() (db *sql.DB, err error) {

      var databaseServer, databaseUser, database, password string
      databaseServer = GetConfigurationParameter("dbserver", "localhost")
      databaseUser = GetConfigurationParameter("dbuser", "")
      database = GetConfigurationParameter("database", "sample")
      password = GetConfigurationParameter("dbpassword", "")
      connectionString := fmt.Sprintf("%v:%v@tcp(%s:3306)/%v?parseTime=true",
            databaseUser, password, databaseServer, database)
      db, err = sql.Open("mysql", connectionString)
      if err != nil {
            WriteLog("Error in SQLConnection: " + err.Error())
      }
      return
}

type UserType struct {
     ID       int
     Username string
}

func ListUsers(db *sql.DB) (users []UserType, err error) {

     sqlStatement := `select id, username from users order by id`
     rows, err := db.Query(sqlStatement)
     users = make([]UserType, 0)
     if err == nil {
          defer rows.Close()
          for rows.Next() {
              var user UserType
              err = rows.Scan(&user.ID, &user.Username)
                  if err == nil {
                         users = append(users, user)
                  }
          }
     } else {
          WriteLog("Error in ListUsers: " + err.Error())
     }
     return
}

func InsertUser(db *sql.DB, username string) (success bool, err error) {

     sqlStatement := `INSERT INTO users (username) values (?)`
     username = strings.TrimSpace(username)
     _, err = db.Exec(sqlStatement, username)
     success = err == nil
     if !success {
          WriteLog("Error in InsertUser:  " + err.Error())
     }
     return
}

At the top of dataaccess.go file, we have two functions: GetConfigurationParameter and WriteLog, these are functions wrappers for similar functions in codeutils package GetConfigValue and WriteToLog. First one reads configuration value from ini file (config.ini) and second one writes event log, such as error message in log file. Calling function wrapper is more easy, convenient, more customized and specific for current project, also we could introduce new features that not exist in original functions of package, such as default value in GetConfigurationParameter function.

There are three database functions:

a. SQLConnection: This connects to MySQL server using parameters in config.ini file, and returns a pointer to database connection object (db *sql.DB)

b. ListUsers: This method retrieves all users information as a list (slice), and it retrieves them from users table.

c. InsertUser: This method inserts new user into users table.

2. controller.go : This Go source file is a higher level than data access layer, it calls dataaceess.go functions and it could add business logic and provides new functions with that business logic for higher level layer (presentation layer), which is in our case main.go.

Here is controller.go code:

package main

import (
    "bufio"
    "database/sql"
    "errors"
    "fmt"
    "os"
    "strings"
)

func readAndInsertUser(db *sql.DB) (err error) {
      
     fmt.Print("Please enter username: ")
     var username string
     in := bufio.NewReader(os.Stdin)
     username, err = in.ReadString('\n')
     if err == nil && strings.TrimSpace(username) == "" 
        err = errors.New("Empty username")
     }
     if err == nil {
         _, err = InsertUser(db, username)
     }
     if err == nil {
        fmt.Println("User: ", username, " Has been added")
     } else {
        fmt.Println("Error in getAndInsertUser: " + err.Error())
     }
    return
}

func showUsers(db *sql.DB) (err error) {
      
     list, err := ListUsers(db)
     if err == nil {
        for _, user := range list {
               fmt.Printf("User #%d: %s\n", user.ID, user.Username)
        }
     } else {
         fmt.Println("Error in showUsers: " + err.Error())
     }
     return
}

func InsertAndShowUsers() {
 
     db, err := SQLConnection()
     if err == nil {
            defer db.Close()
            getAndInsertUser(db)
            showUsers(db)
     }
}