Session 06¶

New topic¶

Topic 2: The Go language¶

Today¶

Tour of Go¶

example: duplicate lines
go tool basics
example: fetching URLs

Go¶

*(Gopher, Go’s maskot)*

Why are we using Go?¶

💬

Go …

is relatively easy to learn & has simple CLI tools
is a compiled language (like C) → fast execution
has concurrency built-in (unlike many other languages)
has a very general ways of expressing parallelism
teaches us most concepts that exist in other languages

Several key factors make Go particularly attractive to developers:

Simplicity and readability are Go’s standout features. The language was deliberately designed with a minimal syntax and small feature set. There’s usually one obvious way to do something, which makes code predictable and easy to understand. This “less is more” philosophy reduces cognitive load and makes large codebases more maintainable.

Exceptional concurrency support through goroutines and channels makes parallel programming much more approachable. Goroutines can be spawned by the thousands without significant overhead, and channels provide a clean way to communicate between them. This makes Go excellent for network services and distributed systems (e.g., cloud and web).

Fast compilation and execution address real developer pain points. Large projects can compile in seconds. The resulting binaries are fast - almost as fast as optimized C code - and have no runtime dependencies, making deployment simpler.

Strong standard library covers most common tasks without external dependencies. HTTP servers, JSON handling, cryptography, and file operations are all built-in and well-designed. This reduces dependency management headaches.

Excellent tooling comes out of the box. go fmt automatically formats code, go test provides built-in testing, go mod handles dependencies, and the toolchain includes profiling and race detection.

Corporate backing and real-world validation from Google gives confidence in the language’s future. It’s used in production for major systems like Docker, Kubernetes, and many high-traffic services.

Learning gobyexample.com ¶

getting to know basic constructs

types, variables, values, constants
importing packages
iterations: for, for range
conditional / selection: if … else, switch
data structures: structs, slices, maps
functions, multiple return values
pointers

→ try out The Go Playground

Tour of Go¶

Go makes CLI programs easy¶

a duplicates program to find duplicate lines …

reads from standard input (👉 terminal pipes)
processes data line by line
uses Go’s built-in data structures
outputs only duplicate lines at the end

cat data.txt | go run duplicates.go  # "|" means pipe

✨ With just a few lines of Go, we will build a fast, reliable tool to processes data streams.

Steps of the duplicates program¶

create a data structure to hold counts of each line
stream line by line from standard input
→ count each line
check for errors from input reader
print lines that occurred more than once

How to count repeated lines in Go¶

Problem:
How do we know if a line is new or repeated?

Elegant solution in Go:
The map data structure represents a dictionary with unique keys!

map[string]int

unique lines of type string → counters of type int

Counting lines is easy in Go¶

// Create a map to hold counts of each line
counts := make(map[string]int)

// LOOP

  // Count a single line
  counts[input.Text()]++

Why this is elegant:

no special libraries
no boilerplate initialization
reads almost like pseudocode

Streaming lines in Go¶

input := bufio.NewScanner(os.Stdin)

Go’s build-in bufio.Scanner is great for large data:

reads input incrementally
constant memory use per line
works with standard input (here), but also with files

→ Don’t guess how big the data is. Stream it.

Hands-on: Get ready to write the program¶

open the terminal, ssh into Matrix
then cd into any folder where you like the code to be
create a new folder for the program & cd into that folder
```
mkdir duplicates
cd duplicates
```

initialize a new Go program

go mod init duplicates  # creates a Go module file `go.mod`

Hands-on: Let’s write the code¶

set up the code file duplicates.go:

// Duplicates prints the text of lines that appear more than once.
package main

import (
    "bufio"
    "fmt"
    "os"
)

func main() {
    // create a data structure to hold counts of each line
    // stream line by line from *standard input*
       // count a single line
    // check for errors from input reader
    // print lines that occurred more than once
}

(more details in code/go/duplicates)

Exercises (more in the README of code/go/duplicates)¶

Ignore empty lines: Modify the program so that empty lines (or lines consisting only of whitespace) are not counted.
Print all lines, not just duplicates: Modify the program so that it prints every distinct line together with its count, including lines that occur only once.

What issues do you see with the new output? How would you fix them?

Go tool¶

Tool design¶

“Go ships with a single command, go, that does everything:
build, test, format, fetch dependencies…

→ Why split across tools when you can have one?

Go tool commands¶

run in the terminal: go <command>

run: builds & runs code
build: build system (compiler, assembler, linker)
→ like gcc + make in C
test: testing system (test, benchmark, profile)
→ like pytest in Python but build-in
fmt: format code → other languages argue about style - Go just enforces it
get/install/list/...: package manager

The go tool combines the features of a diverse set of tools into one command set. It is a package manager (analogous to apt or rpm) that answers queries about its inventory of packages, computes their dependencies, and downloads them from remote version-control systems. It is a build system that computes file dependencies and invokes compilers, assemblers, and linkers, although it is intentionally less complete than the standard Unix make.

Running and building a program¶

cd to location of .go file with main() function
run the code (builds the program automatically)

→ go run .
Hello, world!

build manually and then run

→ go build .
→ ./hello-world
Hello, world!

learn more in the Tutorial: Create a Go module

Formatting code¶

go fmt: formats how the code looks

indentation
placement of spaces, newlines, etc.

→ go help fmt
usage: go fmt [-n] [-x] [packages]

Example: inside the directory of a package

→ go fmt .
# nothing printed, but my code was auto-formatted!

Hidden gems¶

lesser-known but powerful

go vet: source code checker
go doc: built-in docs
go env: shows environment configuration

to list all commands in the terminal: go then <Enter>

(we’re skipping module commands for now)

Tour of Go (continued)¶

Go packages¶

thanks to many useful build-in packages,
so far, we used bufio for streaming from Stdin

—

net is a collection of packages that help us to …

send and receive information through the internet
make low-level network connections
set up servers

From local files to the internet¶

Compare:

os.Stdin → files input (or any user input)
http.Get → internet input

→ In Go, the web is just another data source.

Go makes accessing the web easy¶

a fetch program to fetch URLs …

takes one or more URLs from the command line
downloads HTML file at each URL
prints the response body to standard output

go run fetch.go https://vt.edu

✨ With just a few lines of Go, we will processes data from the web.

Same concept, bigger context¶

`duplicates` program	`fetch` program
standard input	URLs
text files	HTTP
local data	global data

Steps of the fetch program¶

loop over command-line arguments
fetch data from web
handle errors
output HTML

How to download from the web in Go¶

Elegant solution in Go:
Networking is a standard feature, not an add-on.

import "net/http"

resp, err := http.Get(url)  // ... check err

What Go handles for you:

DNS lookup
TCP connection
HTTP request
HTTP response parsing

Reading the response body¶

b, err := io.ReadAll(resp.Body)  // ... check err

resp.Body behaves like a file
ReadAll pulls everything into memory
good for demos and small responses 👆

💡 But we could just as easily stream line by line.
Exactly like before!

Hands-on: Get ready to write the program¶

open the terminal, ssh into Matrix
cd into any folder where you like the code to be
create a new folder for the program & cd into it
```
mkdir fetch
cd fetch
```

initialize a new Go program

go mod init fetch  # creates a Go module file `go.mod`

Hands-on: Create and run the code¶

Fetching URLs - v1 (code/go/fetch/v1)¶

import the package net/http (along with fmt, io, os)
overview of the program:

//func main()

  for _, url := range os.Args[1:] {
    resp, err := http.Get(url)  //.. check err
    b, err := io.ReadAll(resp.Body)  //.. check err, defer close
    fmt.Printf("%s", b)
  }

gopl.io, p16:

This program introduces functions from two packages, net/http and io. The http.Get function makes an HTTP request and, if there is no error, returns the result in the response struct resp. The Body field of resp contains the server response as a readable stream. Next, io.ReadAll reads the entire response; the result is stored in b. The Body stream is closed to avoid leaking resources, and Printf writes the response to the standard output.

(more details in code/go/fetch)

Note: Read all vs. streaming¶

instead of:

b, err := io.ReadAll(resp.Body)  // ... check err
...
fmt.Printf("%s", b)

we can attach a bufio.Scanner to resp.Body
(just like before with os.Stdin):

// The web is just another stream.
input := bufio.NewScanner(resp.Body)
for input.Scan() {
    fmt.Println(input.Text())
}
if err := input.Err(); err != nil {
    log.Fatalf("reading %q: %v", url, err)
}

Fetching URLs - v2 (code/go/fetch/v2)¶

fetch URL and save to file

//func main()

  for _, url := range os.Args[1:] {
    resp, err := http.Get(url)  //.. check err, defer close
    filename := sanitizeFilename(url)
    out, err := os.Create(filename)  //.. check err, defer close
    _, err := io.Copy(out, resp.Body)  //.. check err
    log.Printf("saved %q to %q", url, filename)
  }

Example implementation of the sanitize function (it needs strings to be imported):

func sanitizeFilename(url string) string {
    f := strings.ReplaceAll(url, "https://", "")
    f = strings.ReplaceAll(f, "/", "_")
    f = strings.ReplaceAll(f, ":", "_")
    return f
}

Exercises (more in the README of code/go/fetch)¶

Print the HTTP status code: Modify the program so that it logs the HTTP response status code (e.g. 200 OK) for each URL (consider resp.Status).
Add support for http:// URLs: Modify sanitizeFilename so that it also correctly handles URLs starting with http://.
Change the filename format: Modify the program so that files are saved with a .html extension, regardless of the URL.
Modify fetch to add the prefix http:// to each argument URL if it is missing. You might want to use strings.HasPrefix.

Key takeaways¶

~40 lines of Go get us:

file streaming / HTTP clients
command-line tools
proper error handling
clean resource management

Data science applications:

→ data ingestion, web scraping, data pipelines

Looking ahead¶

continue the tour of Go
graphics
web servers

Prepare for the next session 🌱¶

sessions/07-todo

Session 06¶

New topic¶

Topic 2: The Go language¶

Today¶

Tour of Go¶

Go¶

Why are we using Go?¶

Learning gobyexample.com¶

Tour of Go¶

Go makes CLI programs easy¶

Steps of the duplicates program¶

How to count repeated lines in Go¶

Counting lines is easy in Go¶

Streaming lines in Go¶

Hands-on: Get ready to write the program¶

Hands-on: Let’s write the code¶

Exercises (more in the README of code/go/duplicates)¶

Go tool¶

Tool design¶

Go tool commands¶

Running and building a program¶

Formatting code¶

Hidden gems¶

Tour of Go (continued)¶

Go packages¶

From local files to the internet¶

Go makes accessing the web easy¶

Same concept, bigger context¶

Steps of the fetch program¶

How to download from the web in Go¶

Reading the response body¶

Hands-on: Get ready to write the program¶

Hands-on: Create and run the code¶

Fetching URLs - v1 (code/go/fetch/v1)¶

Note: Read all vs. streaming¶

Fetching URLs - v2 (code/go/fetch/v2)¶

Exercises (more in the README of code/go/fetch)¶

Key takeaways¶

Looking ahead¶

Prepare for the next session 🌱¶

Learning gobyexample.com ¶