golang knowledge

[Yu-Jack]

預計整理以下內容變成一篇新手學 golang 注意到的地方

Q1 - variable scope in for statement
Q2 - Interface & Receiver
Q3 - ServerMux trailiing slash
Q4 - Internal package
Q5 - Double Pointer
Q6 - Callback function
Q7 - buffered channel & unbuffered channel
Q8 - Three ways to sync flow
Q9 - gomock some issuses
Q10 - init
Q11 - channel select with default
Q12 - make & new
Q13 - Defer
Q17 - context.Context
Q19 - range with channel
Q20 - heap & stack in memory performance (not data structure heap & stack)

Other 01 - How to set up the protocol buffers file paths in GoLand.

https://william-yeh.net/post/2020/07/golang-best-practices/

[Yu-Jack]

Q1 - variable scope in for statement

package main

import (
	"fmt"
)

type student struct {
	Name string
	Age  int
}

func main() {
	m := make(map[string]*student)
	stus := []student{
		{Name: "zhou", Age: 24},
		{Name: "li", Age: 23},
		{Name: "wang", Age: 22},
	}
	for _, stu := range stus {
		m[stu.Name] = &stu
	}
	for k, v := range m {
		fmt.Println(k, "=>", v.Name) // All the value of k will point to the same value.
	}
}

By printing the address, you'll find they're the same.

for _, stu := range stus {
	fmt.Printf("%p\n", &stu)
	m[stu.Name] = &stu
}

According to documentation - For_statements.

The iteration variables may be declared by the "range" clause using a form of short variable declaration (:=). In this case their types are set to the types of the respective iteration values and their scope is the block of the "for" statement; they are re-used in each iteration. If the iteration variables are declared outside the "for" statement, after execution their values will be those of the last iteration.

So, every time the value of stu will be changed like below, but the address is the same because of reusing the variable.

A (address of stu) --> zhou (value of stu)
A (address of stu) --> li (value of stu)
A (address of stu) --> wang (value of stu)

You should change your code to following example by assigning [value of stu] to temp variable or using index.
But, it's different address in the m.

for _, stu := range stus {
	// method1
	temp_stu := stu
	m[stu.Name] = &temp_stu // address of local var

	// method2
	m[stus[i].Name] = &stus[i] // address of slices
}

[Yu-Jack]

Q2 - Interface & Receiver

Q: Could this compile successfully?

package main

import (
	"fmt"
)

type People interface {
	Speak(string) string
}

type Stduent struct{}

func (stu *Stduent) Speak(think string) (talk string) {
	if think == "jack" {
		talk = "You are a good guy"
	} else {
		talk = "hi"
	}
	return
}

func main() {
	var peo People = Stduent{}
	think := "jack"
	fmt.Println(peo.Speak(think))
}

A: No.

Because the struct of *Student (pointer type) has method Speak, that doesn't mean the struct of Student (value type) has this method. So, the the struct of Student doesn't implement Speak which is defined in People interface.
We could use two ways to modify this code.

1. Use & to assign Student to peo like var peo People = &Student{}.
   (Go will interpret it to (*peo).Speak(think) when it's executed )
2. Remove the * in Speak function. (Implement method the value type of Student)

Basically when you use pointer receiver, your expectation is that change the value, not copy the value.
If Golang allowed you assign value type to interface value when you implement pointer receiver, it will not change value.

type Changer interface {
	UpperCaseName ()
}

type User struct {
	Name string
}

func (u *User) UpperCaseName()  {
	u.Name = strings.ToUpper(u.Name)
}

func main() {
	user := User{Name: "miles"}
	var c Changer = user  // -> if this line is allowed,
	c.UpperCaseName()
	fmt.Println(user.Name) // this name is still lowercase "miles", but it's not our expectation.
}

passed by value

When you assign the value to another variable without passing pointer, it just do the copy. There is only passed by value in go). But, Map and slice values behave like pointers, even embedded in struct. For assigning new variable In below example, it do the copy, so the value of ttt is empty, not "qwe".

package main
import "fmt"
type Stduent struct {
	Name string
}
func (stu *Stduent) Change(think string) {
	stu.Name = think
}
func main() {
	var stu = Stduent{}
	ttt := stu
	stu.Change("qwe")
	fmt.Println(ttt)
}

More examples

package main

import "fmt"

type data struct {
	Name string
}

func NewData() data {
	return data{
		Name: "Jack",
	}
}
func (data *data) changeName() {
	data.Name = "Fixed"
}
func main() {
	d := NewData()
	fmt.Printf("Type is %T, value is %+v, address is %p\n", &d, &d, &d)
	d.changeName()
	fmt.Printf("Type is %T, value is %+v, address is %p\n", &d, &d, &d)
}
// Type is *main.data, value is &{Name:Jack}, address is 0xc000010240
// Type is *main.data, value is &{Name:Fixed}, address is 0xc000010240

You could see we use NewData to retrieve a data struct back.
Then using point receiver to change name, it looks great.

But, does it return a whole new copied value to us in NewData function?

func NewData() data {
	return data{
		Name: "Jack",
	}
}

We could rewrite this function like below.

func NewData() data {
	return data{
		Name: "Jack",
	}
}

Then using following code to run again.

package main

import "fmt"

type data struct {
	Name string
}

func NewData() data {
	d := data{
		Name: "Jack",
	}
	fmt.Printf("Inside: Type is %T, value is %+v, address is %p\n", &d, &d, &d)
	return d
}
func (data *data) changeName() {
	data.Name = "Fixed"
}
func main() {
	d := NewData()
	fmt.Printf("Outside Type is %T, value is %+v, address is %p\n", &d, &d, &d)
	d.changeName()
	fmt.Printf("Outside Type is %T, value is %+v, address is %p\n", &d, &d, &d)
}
// Inside: Type is *main.data, value is &{Name:Jack}, address is 0xc000010250
// Outside Type is *main.data, value is &{Name:Jack}, address is 0xc000010240
// Outside Type is *main.data, value is &{Name:Fixed}, address is 0xc000010240

You'll see the different address, so does the inside data change?
No, you could use global variable to check it.

package main

import "fmt"

type data struct {
	Name string
}

var _d = data{
	Name: "Jack",
}

func PrintData() {
	fmt.Printf("PrintData: Type is %T, value is %+v, address is %p\n", &_d, &_d, &_d)
}
func NewData() data {
	fmt.Printf("Inside: Type is %T, value is %+v, address is %p\n", &_d, &_d, &_d)
	return _d
}
func (data *data) changeName() {
	data.Name = "Fixed"
}
func main() {
	d := NewData()
	fmt.Printf("Outside Type is %T, value is %+v, address is %p\n", &d, &d, &d)
	d.changeName()
	fmt.Printf("Outside Type is %T, value is %+v, address is %p\n", &d, &d, &d)
	PrintData()
}
// Inside: Type is *main.data, value is &{Name:Jack}, address is 0x1159c80
// Outside Type is *main.data, value is &{Name:Jack}, address is 0xc000096220
// Outside Type is *main.data, value is &{Name:Fixed}, address is 0xc000096220
// PrintData: Type is *main.data, value is &{Name:Jack}, address is 0x1159c80

So, it means that we got the two copied data struct in this program, but it's not good.
We need to change above code a little bit.
We need to let NewData function to return the pointer, so that it will make data struct the same address.

var _d = &data{
	Name: "Jack",
}
func NewData() *data {
	fmt.Printf("Inside: Type is %T, value is %+v, address is %p\n", _d, _d, _d)
	return _d
}

The whole code is like this.

package main

import "fmt"

type data struct {
	Name string
}

var _d = &data{
	Name: "Jack",
}

func PrintData() {
	fmt.Printf("PrintData: Type is %T, value is %+v, address is %p\n", _d, _d, _d)
}
func NewData() *data {
	fmt.Printf("Inside: Type is %T, value is %+v, address is %p\n", _d, _d, _d)
	return _d
}
func (data *data) changeName() {
	data.Name = "Fixed"
}
func main() {
	d := NewData()
	fmt.Printf("Outside Type is %T, value is %+v, address is %p\n", d, d, d)
	d.changeName()
	fmt.Printf("Outside Type is %T, value is %+v, address is %p\n", d, d, d)
	PrintData()
}
// Inside: Type is *main.data, value is &{Name:Jack}, address is 0x1159c80
// Outside Type is *main.data, value is &{Name:Jack}, address is 0x1159c80
// Outside Type is *main.data, value is &{Name:Fixed}, address is 0x1159c80
// PrintData: Type is *main.data, value is &{Name:Fixed}, address is 0x1159c80

After rewriting the above code, we'll get this.

package main

import "fmt"

type data struct {
	Name string
}

func NewData() *data {
	return &data{
		Name: "Jack",
	}
}
func (data *data) changeName() {
	data.Name = "Fixed"
}
func main() {
	d := NewData()
	fmt.Printf("Outside Type is %T, value is %+v, address is %p\n", d, d, d)
	d.changeName()
	fmt.Printf("Outside Type is %T, value is %+v, address is %p\n", d, d, d)
}

It's useful when you want to use same struct in whole program to reduce the memory.

Restrict to use pointer type with interface intentionally

But, we could use above point to restrict that we "must assign pointer" to reduce the memory cost when using struct.
You could use interface to restrict this situation just like the answer 1 in this post.

package main

type data struct {
	Name string
}
type Data interface {
	changeName()
}

func NewData() Data {
	return &data{ // if you remove this &, you'll get error message like the first paragraph in this post
		Name: "Jack",
	}
}
func (data *data) changeName() {
	data.Name = "Fixed"
}
func main() {
	d := NewData()
	d.changeName()
}

References

1. https://golang.org/doc/faq#Functions_methods
2. https://tour.golang.org/methods/6
3. https://golang.org/ref/spec#Struct_types
4. https://stackoverflow.com/questions/40823315/x-does-not-implement-y-method-has-a-pointer-receiver
5. https://golang.org/ref/spec#Method_sets
6. https://mileslin.github.io/2020/08/Golang/%E7%82%BA%E4%BB%80%E9%BA%BC-Pointer-Receiver-%E4%B8%8D%E8%83%BD%E4%BD%BF%E7%94%A8-Value-Type-%E8%B3%A6%E5%80%BC%E7%B5%A6-Interface-Value/

[Yu-Jack]

Q3 ServerMux trailiing slash

if you registered for both "/images/" and "/images/thumbnails/", the latter will be called when you make api path for "/images/thumbnails/", the previous will be called when you make api path for "/images/" or "/images/123".

When you registered "/images", it will not be called when you make api for "/images/".
But, if you registered "/images/", it will be called when you make api for "/images/" or "/images".

Otherwise, you could registered separated path "/images" and "/images"/.
"/images" will be called when you make api for "/images"
"/images/" will be called when you make api for "/images/"

More examples

// 1.
mux.HandleFunc("/todo-list/get", ...)
// curl http://localhost:7070/todo-list/get will trigger this.

// 2.
mux.HandleFunc("/todo-list/get/", ...)
// curl http://localhost:7070/todo-list/get/ will trigger this.
// curl http://localhost:7070/todo-list/get/1234 will trigger this.
// curl http://localhost:7070/todo-list/get/124 will trigger this.

// 3.
mux.HandleFunc("/todo-list/get/123", ...)
// curl http://localhost:7070/todo-list/get/123 will trigger this.

References

1. https://golang.org/pkg/net/http/#ServeMux

[Yu-Jack]

Q4 - Internal package

In Go 1.4 release note, there is a new mechanism about package.

For example, a package .../a/b/c/internal/d/e/f can be imported only by code in the directory tree rooted at .../a/b/c. It cannot be imported by code in .../a/b/g or in any other repository.

If you try to import /a/b/c/internal/d/e/f from /a/b/g, go will show use of internal package not allowed error message. Usually, if you don't want your package be used in other place, you could create internal folder to avoid it.

[Yu-Jack]

Q5 - Double Pointer

// taking a variable
// of integer type
var V int = 100
fmt.Println("The Value of Variable V is = ", V)
fmt.Println("Address of variable V is = ", &V)
// ---
// The Value of Variable V is =  100
// Address of variable V is =  0xc000118000

var pt1 *int = &V
fmt.Println("The Value of pt1 is = ", pt1)
fmt.Println("Address of pt1 is = ", &pt1)
fmt.Println("Dereference value of pt1 is = ", *pt1)
// ---
// The Value of pt1 is =  0xc000118000
// Address of pt1 is =  0xc00010e020
// Dereference value of pt1 is =  100

var pt2 **int = &pt1
fmt.Println("The value of pt2 is = ", pt2)
fmt.Println("Value at the address of pt2 is or *pt2 = ", *pt2)
fmt.Println("*(Value at the address of pt2 is) or **pt2 = ", **pt2)
// ---
// The value of pt2 is =  0xc00010e020
// Value at the address of pt2 is or *pt2 =  0xc000118000
// *(Value at the address of pt2 is) or **pt2 =  100

So, you could use * to dereference to get the value. When you deference and assign the new value, you also change the original value, see below example.

// taking a variable
// of integer type
var v int = 100
fmt.Println("The Value of Variable v is = ", v)

// taking a pointer
// of integer type
var pt1 *int = &v
// changing the value of v by assigning
// the new value to the pointer pt1
*pt1 = 200

fmt.Println("Value stored in v after changing pt1 = ", v)
// ---
// The Value of Variable v is =  100
// Value stored in v after changing pt1 =  200

https://www.geeksforgeeks.org/go-pointer-to-pointer-double-pointer/

[Yu-Jack]

Q6 - Callback function

package main

import "fmt"

func test(f func(input string)) {
	f("hi")
}

func main() {
	callback := func(input string) {
		fmt.Println(input)
	}

	test(callback)
}

[Yu-Jack]

Q7 - buffered channel & unbuffered channel

Before diving into the channel, we pick up a one example to understand the execution order. If run the following program, it doesn't print the hi in the result. Cause the main program is finished.

package main

import "fmt"

func main() {
	go func() {
		fmt.Println("hi")
	}()
}

If you add time.Sleep(), you could see the hi in the result cause of time.Sleep.

package main

import (
	"fmt"
	"time"
)

func main() {
	go func() {
		fmt.Println("hi")
	}()
	time.Sleep(2 * time.Second)
}

But, in reality we don't add time.Sleep, so we'll use channel to control the flow like following example which prints the got it in the result.

package main

import (
	"fmt"
)

func main() {
	ch := make(chan string)
	go func() {
		fmt.Println("got it")
		<-ch
	}()
	ch <- "hi"
}

But above example is the unbuffered channel, that means you need to finish read and write operation both, then the main program will exit. In other words, if there is a value in unbuffered channel, it must be read out then main program could exit. But if you use buffered channel, the value in buffered channel is not need to be read out then main program could exit. Try following example, it will exit without result.

package main

import (
	"fmt"
)

func main() {
	ch := make(chan string, 1)
	go func() {
		fmt.Println("got it")
		<-ch
	}()
	ch <- "hi"
}

So, this is an important difference between buffered channel and unbuffered channel. But, if you change the <-ch and ch <- "hi" to following example, both generate the same result. Because both want to read value from channel, so both will wait for the value.

package main

import (
	"fmt"
)

func main() {
	ch := make(chan string) // or ch := make(chan string, 2)
	go func() {
		fmt.Println("got it")
		ch <- "hi"
	}()
	<-ch
}

And we could go deeper, if we remove the goroutine and use unbuffered channel, what happen?

package main

import "fmt"

func main() {
	c := make(chan string)
	c <- "1"
	<-c
	fmt.Println("finished")
}
// fatal error: all goroutines are asleep - deadlock!

It'll throw fatal error. Because there is not available goroutine which could read value from the unbuffered channel, so the whole program blocks. But situation is changed when using buffered channel cause the buffered channel don't need to be read out.

package main

import "fmt"

func main() {
	c := make(chan string, 1)
	c <- "1"
	<-c
	fmt.Println("finished")
}
// 1
// finished

chan in function parameter

only send channel

package main

import "fmt"

func main() {
	ch := make(chan int, 3)
	process(ch)
	fmt.Println(<-ch)
}
func process(ch chan<- int) {
	ch <- 2
}

only receive channel

package main

import "fmt"

func main() {
	ch := make(chan int, 3)
	ch <- 2
	process(ch)
}
func process(ch <-chan int) {
	fmt.Println(<-ch)
}

https://easonwang.gitbook.io/golang/ji-ben-yu-fa/channel/buffered-channel-yu-unbuffered-channel
https://blog.wu-boy.com/2019/04/understand-unbuffered-vs-buffered-channel-in-five-minutes/
https://gobyexample.com/channel-synchronization

[Yu-Jack]

Q8 - Three ways to sync flow

1. Mutex (focus on shared data or control flow)

The following program will write "cool-1" or "cool-2" or "cool-1cool-2" or "cool-2cool1" into a.txt file because both read / write the file. You cant expect the result.

package main

import (
	"fmt"
	"io/ioutil"
	"sync"
	"time"
)

func main() {

	go func() {
		fmt.Println("cool-1 is running")
		data, _ := ioutil.ReadFile("a.txt")
		data = append(data, []byte("cool-1")...)
		ioutil.WriteFile("a.txt", data, 0644)
	}()

	go func() {
		fmt.Println("cool-2 is running")
		data, _ := ioutil.ReadFile("a.txt")
		data = append(data, []byte("cool-2")...)
		ioutil.WriteFile("a.txt", data, 0644)
	}()

	time.Sleep(2 * time.Second)
}

So you could use sync.Mutex to control the resource permission of usage like below. It will stably write "cool-1cool-2" into a.txt file.

package main

import (
	"fmt"
	"io/ioutil"
	"sync"
	"time"
)

var lock sync.Mutex

func main() {

	go func() {
		fmt.Println("cool-1 is running")
		lock.Lock()
		fmt.Println("get lock-1")
		data, _ := ioutil.ReadFile("a.txt")
		data = append(data, []byte("cool-1")...)
		ioutil.WriteFile("a.txt", data, 0644)
		lock.Unlock()
		fmt.Println("unlock lock-1")
	}()

	go func() {
		fmt.Println("cool-2 is running")
		lock.Lock()
		fmt.Println("get lock-2")
		data, _ := ioutil.ReadFile("a.txt")
		data = append(data, []byte("cool-2")...)
		ioutil.WriteFile("a.txt", data, 0644)
		lock.Unlock()
		fmt.Println("unlock lock-2")
	}()

	time.Sleep(2 * time.Second)
}

2. Channel

package main

import (
	"fmt"
)

func main() {
	ch := make(chan int)
	done := make(chan bool)

	go func() {
		for i := 1; i <= 5; i++ {
			ch <- i
		}
		close(ch)
	}()

	go func() {
		for i := range ch {
			fmt.Println(i)
		}
		done <- true
	}()

	<-done
}

3. WaitGroup

package main

import (
	"fmt"
	"sync"
)

func main() {
	ch := make(chan int)

	wg := sync.WaitGroup{}
	wg.Add(2)

	go func() {
		for i := 1; i <= 5; i++ {
			ch <- i
		}
		close(ch)
		wg.Done()
	}()

	go func() {
		for i := range ch {
			fmt.Println(i)
		}
		wg.Done()
	}()

	wg.Wait()
}

[Yu-Jack]

Q9 - gomock some issuses

When using gomock.Eq with map[string]interface{}, we should pay attention to integer type. Like int32(0) == 0 is true, but if you use gomock.Eq, it will be failed cause of type is wrong. Because gomock.Eq use reflect.DeepEqual to judge.

[Yu-Jack]

Q10 - init

init function will be executed before the main function, after the initialization.

package main

import "fmt"

var a int = 1

func init() {
	a = 3
}

func main() {
	fmt.Println(a)
}
// 3

If there are multiple init functions, they will be executed by order of declaration.

package main

import "fmt"

func init() {
	fmt.Println("1")
}
func init() {
	fmt.Println("2")
}
func init() {
	fmt.Println("3")
}

func main() {
	fmt.Println("main")
}
// 1
// 2
// 3
// main

[Yu-Jack]

Q11 - channel select with default

[Yu-Jack]

Q12 - make & new

1. new returns address, but make doesn't.
2. make allocates the address and initialize the value, but new only returns address.
3. make is used in slice, map and channel

https://blog.wu-boy.com/2021/06/what-is-different-between-new-and-make-in-golang/

[Yu-Jack]

Q13 - Defer

Defer is FILO. It will be executed in the end of the function.

package main

import "fmt"

func pr(st string) {
	fmt.Println("defer-" + st)
}

func main() {
	defer pr("1")
	defer pr("2")
	defer pr("3")
	fmt.Println("1")
	fmt.Println("2")
	fmt.Println("3")
}
// 1
// 2
// 3
// defer-3
// defer-2
// defer-1

So, if you use it in for-loop, it still be executed in the end of function, not for-loop.

package main

import "fmt"

func pr(st string) {
	fmt.Println("defer-" + st)
}

func main() {
	for _, i := range []string{"1", "2"} {
		defer pr(i)
		fmt.Println(i)
	}
	fmt.Println("for-loop done")
}
// 1
// 2
// for-loop done
// defer-2
// defer-1

And for example, if you want to calculate the function execution time, you could use defer and make your function return new function, then it'll wrap your caller function like below.

package main

import "fmt"

func cool() func() {
	fmt.Println("cool")
	return func() {
		fmt.Println("cool-2")
	}
}

func main() {
	defer cool()()
	fmt.Println("hi")
}
// cool
// hi
// cool-2