Understanding Cgo a little better  

Go is a language that has become popular for a lot of different purposes. It’s being used by back-end developers, who use it to build websites and services ; it’s being used by system engineers, who use it to interact with low-level APIs ; it’s, in general, getting a lot of love for its simplicity and straightforwardness.

One of the features that made Go popular with lower-level developers is Cgo. With Cgo, developers can link their binaries to C libraries - which spares them the expense of having to rewrite specific low-level libraries written in C.
At The Things Network, we use Cgo to interface our own Go-based packet forwarder with the loragw C library written by Semtech, used to exchange packets with a LoRa concentrator. This article is for you if you’re looking for reasons why your Cgo-interfaced program might not work as you expect it to, or if you want to know more on the insides of Cgo.

 Using Cgo

To use C code in your Go, enable Cgo by calling import "C", and call C code with the C. prefix. You can add C standard libraries and C functions by adding them above of import "C":

package main

// #include <stdio.h>
// void print_str(char * valid_str) {
//   printf("%s\n", valid_str);
// }
import "C"

func main() {
    t := "Hello world!"
    C.print_str(C.CString(t))
}

You can also import your own libraries by passing specific flags:

package main

// #cgo CFLAGS: -I${SRCDIR}/../c_library/inc
// #cgo LDFLAGS: -lm ${SRCDIR}/../c_library/my_library.a
// #include "my_library.h"
import "C"

func main() {
    t := "Hello world!"
    C.my_print(C.CString(t))
}

 How does the Go compiler interpret the C parts?

 Pre-processing

Using Cgo requires you to respect the C typing in your code. If you change the types in your code, you’ll notice the Go compiler cannot proceed to compilation ; for example, for this code:

package main

// #include <stdio.h>
// void print_str(char * valid_str) {
//   printf("%s\n", valid_str);
// }
import "C"

func main() {
    var t2 int
    C.print_str(C.int(t2)) // calling a C.int where a C.CString should be called
}

The compiler fails:

$ go build main.go
# command-line-arguments
./main.go:11: cannot use C.int(t2) (type C.int) as type *C.char in argument to _Cfunc_print_str

To make sure the given types are correct, the Go compiler creates preliminary C files, that serve only the purpose of type-checking, and tries to pre-process them with gcc. If the pre-processing fails, the Go compiler parses gcc‘s output to deduce the errors. Generating and compiling C code to type-check is much less expensive to maintain than understanding ELF - which C compilers already do very well.

 Code generation

When the code has been pre-processed, now’s the time when the Go compiler starts linking the two of them. It starts by generating two kind of files:

Let’s have a proper look at the Go workbench, where multiple files are generated. When compiling Go code, you can keep the workbench by passing the -work flag to go build.

These are Go files:

// main.cgo1.go

[...]

//line /Users/egourlao/code/blog/main.go:11
func main() {
    t := "Hello world!"
    _Cfunc_print_str(_Cfunc_CString(t))
    fmt.Println("vim-go")
}
// _cgo_gotypes.go

[...]

//go:cgo_import_static _cgo_2a24a2eca554_Cfunc_print_str
//go:linkname __cgofn__cgo_2a24a2eca554_Cfunc_print_str _cgo_2a24a2eca554_Cfunc_print_str
var __cgofn__cgo_2a24a2eca554_Cfunc_print_str byte
var _cgo_2a24a2eca554_Cfunc_print_str = unsafe.Pointer(&__cgofn__cgo_2a24a2eca554_Cfunc_print_str)

//go:cgo_unsafe_args
func _Cfunc_print_str(p0 *_Ctype_char) (r1 _Ctype_void) {
    _cgo_runtime_cgocall(_cgo_2a24a2eca554_Cfunc_print_str, uintptr(unsafe.Pointer(&p0)))
    if _Cgo_always_false {
        _Cgo_use(p0)
    }
    return
}

In main.cgo2.c, the Go compiler has generated C code, that matches both the function we’ve written above, and another function, capable of fetching the parameters we passed to the Go code, and to pass them to the C function:

void print_str(char * valid_str) {
  printf("%s\n", valid_str);
}

[...]

void
_cgo_2a24a2eca554_Cfunc_print_str(void *v)
{
    struct {
        char* p0;
    } __attribute__((__packed__)) *a = v;
    _cgo_tsan_acquire();
    print_str(a->p0);
    _cgo_tsan_release();
}

 Linking

Here’s the part where Cgo reveals to be tricky. The Cgo calls we saw in the Go call earlier aren’t linked to the functions at compilation - Cgo creates a first dynamic executable with the C code, and with an empty int main() { return 0; }:

// _cgo_main.c
int main() { return 0; }
void crosscall2(void(*fn)(void*, int, __SIZE_TYPE__), void *a, int c, __SIZE_TYPE__ ctxt) { }
__SIZE_TYPE__ _cgo_wait_runtime_init_done() { return 0; }
void _cgo_release_context(__SIZE_TYPE__ ctxt) { }
char* _cgo_topofstack(void) { return (char*)0; }
void _cgo_allocate(void *a, int c) { }
void _cgo_panic(void *a, int c) { }
void _cgo_reginit(void) { }

Cgo then examines this executable to find the paths to the shared object, and uses them to compile the Go binary. This way, Cgo doesn’t have to parse the .o object file.

 Runtime

Finally, the runtime/cgo package import is added to the Go code - this initiates a few runtime variables, to indicate that Cgo is running. This changes the behaviour of a package at runtime. For example, threads are more expensive when using Cgo, and the behaviour of a blocking call is not the same.

 
3
Kudos
 
3
Kudos

Now read this

Using Docker as a basic Node developing environment

For the past few days, I’ve been working on this small NodeJS project. Node is a very powerful and fast JavaScript runtime environment, and it’s gotten a lot of attention these last few years as a credible back-end for Web... Continue →