Read the original article
Go is a statically compiled language that gained a lot of popularity lately due to the fact that is simple, performant and fits really well with developing cloud applications. It has a strong, yet poorly documented sub-package level base library that deals with a lot of aspects related to internationalization (i18n) and localization (l10n), such as character encodings, text transformations, and locale-specific text handling. Let's see what we can do to master this library and make our Go applications locale aware.
golang.org/x/textgolang.org/x/textFor the purposes of this tutorial, I will be using the latest Go v1.10 and the code for this tutorial is hosted on Github.
Let’s get going.
Overview Of The Package
fmtgolang.org/x/textMessages And Catalogs
A message is some form of content to be conveyed to the user. Each message is identified by a key, which can have many forms. You can create a message printer like that:
1p := message.NewPrinter(language.BritishEnglish)
2p.Printf("There are %v flowers in our garden.", 1500)You need to supply a Language Tag when you call the NewPrinter function. Language tags are used whenever you want to specify a language. There are many ways you can create a tag such as:
- Using predefined tags. For example:
1language.Greek, language.BrazilianPortugueseThe whole list of predefined tags is listed here.
- From a string value. For example:
1language.Make("el"), language.Parse("en-UK")- By composing parts of type Tag, Base, Script, Region, Variant, []Variant, Extension, []Extension or error. For example:
1ja, _ := language.ParseBase("ja")
2jp, _ := language.ParseRegion("JP")
3jpLngTag, _ := language.Compose(ja, jp)
4fmt.Println(jpLngTag) // prints ja-JPIf you specify an invalid language tag you will get an instance of the Und Tag which denotes an Undefined Tag.
1fmt.Println(language.Compose(language.ParseRegion("AL"))) // prints Und-ALf you want to learn more about the language API seethis doc here.
Coming back to our messages we can assign a new printer using a different language and print the formatted strings. The library will take care any localized formatting variants for you:
1package main
2import (
3 "golang.org/x/text/message"
4 "golang.org/x/text/language"
5)
6func main() {
7 p := message.NewPrinter(language.BritishEnglish)
8 p.Printf("There are %v flowers in our garden.\n", 1500)
9 p = message.NewPrinter(language.Greek)
10 p.Printf("There are %v flowers in our garden.", 1500)
11}If you run this program you will get:
1$ go run main.go
2There are 1,500 flowers in our garden.
3There are 1.500 flowers in our garden.Now in order to print translated messages, we need to add them to the message catalog so that the Printer can find them for the right language tag.
A Catalog defines collections of translated format strings. Think of it as a set of per-language dictionaries with translations for a set of keys. In order to use catalogs, we need to populate them with translations.
In practice, translations will be automatically injected from a translator-supplied data source. Let’s see how we can do it manually:
1package main
2import (
3 "golang.org/x/text/message"
4 "golang.org/x/text/language"
5 "fmt"
6)
7func init() {
8 message.SetString(language.Greek, "%s went to %s.", "%s πήγε στήν %s.")
9 message.SetString(language.AmericanEnglish, "%s went to %s.", "%s is in %s.")
10 message.SetString(language.Greek, "%s has been stolen.", "%s κλάπηκε.")
11 message.SetString(language.AmericanEnglish, "%s has been stolen.", "%s has been stolen.")
12 message.SetString(language.Greek, "How are you?", "Πώς είστε?.")
13}
14func main() {
15 p := message.NewPrinter(language.Greek)
16 p.Printf("%s went to %s.", "Ο Πέτρος", "Αγγλία")
17 fmt.Println()
18 p.Printf("%s has been stolen.", "Η πέτρα")
19 fmt.Println()
20 p = message.NewPrinter(language.AmericanEnglish)
21 p.Printf("%s went to %s.", "Peter", "England")
22 fmt.Println()
23 p.Printf("%s has been stolen.", "The Gem")
24}If you run this program you will get the following output:
1$ go run main.go
2Ο Πέτρος πήγε στήν Αγγλία.
3Η πέτρα κλάπηκε.
4Peter is in England.
5The Gem has been stolen.%SetStringPrintLn\n1p := message.NewPrinter(language.Greek)
2p.Printf("%s went to %s.\n", "Ο Πέτρος", "Αγγλία") // will print Ο Πέτρος went to Αγγλία.
3p.Println("How are you?") // will print How are you?Typically you don’t create catalogs but let the library handle them for you. You can also have the option to build ones programmatically using the catalog.Builder function.
Handling Plurals
golang.org/x/text/feature/pluralI give below some typical usages of this function:
1func init() {
2 message.Set(language.Greek, "You have %d. problem",
3 plural.Selectf(1, "%d",
4 "=1", "Έχεις ένα πρόβλημα",
5 "=2", "Έχεις %[1]d πρόβληματα",
6 "other", "Έχεις πολλά πρόβληματα",
7 ))
8 message.Set(language.Greek, "You have %d days remaining",
9 plural.Selectf(1, "%d",
10 "one", "Έχεις μία μέρα ελεύθερη",
11 "other", "Έχεις %[1]d μέρες ελεύθερες",
12 ))
13}
14func main() {
15 p := message.NewPrinter(language.Greek)
16 p.Printf("You have %d. problem", 1)
17 fmt.Println()
18 p.Printf("You have %d. problem", 2)
19 fmt.Println()
20 p.Printf("You have %d. problem", 5)
21 fmt.Println()
22 p.Printf("You have %d days remaining", 1)
23 fmt.Println()
24 p.Printf("You have %d days remaining", 10)
25 fmt.Println()
26}If you run this program you will get the following output:
1$ go run main.go
2Έχεις ένα πρόβλημα
3Έχεις 2 πρόβληματα
4Έχεις πολλά πρόβληματα
5Έχεις μία μέρα ελεύθερη
6Έχεις 10 μέρες ελεύθερεςzeroonetwofewmany>xString Interpolation In Messages
In some other cases where you want to handle further possible variants of a message, you can assign placeholder variables that can handle some specific cases of linguistic features. For instance, in the previous example where we used the plural can be written as:
1func init() {
2 message.Set(language.Greek, "You are %d minute(s) late.",
3 catalog.Var("minutes", plural.Selectf(1, "%d", "one", "λεπτό", "other", "λεπτά")),
4 catalog.String("Αργήσατε %[1]d ${minutes}."))
5}
6func main() {
7 p := message.NewPrinter(language.Greek)
8 p.Printf("You are %d minute(s) late.", 1) // prints Αργήσατε 1 λεπτό
9 fmt.Println()
10 p.Printf("You are %d minute(s) late.", 10)// prints Αργήσατε 10 λεπτά
11 fmt.Println()
12}catalog.Varminutes%dFormatting Currency
golang.org/x/text/currency 1package main
2import (
3 "golang.org/x/text/message"
4 "golang.org/x/text/language"
5 "fmt"
6 "golang.org/x/text/currency"
7)
8func main() {
9 p := message.NewPrinter(language.English)
10 p.Printf("%d", currency.Symbol(currency.USD.Amount(0.1)))
11 fmt.Println()
12 p.Printf("%d", currency.NarrowSymbol(currency.JPY.Amount(1.6)))
13 fmt.Println()
14 p.Printf("%d", currency.ISO.Kind(currency.Cash)(currency.EUR.Amount(12.255)))
15 fmt.Println()And the result will be:
1$ go run main.go
2US$ 0.10
3¥ 2
4EUR 12.26Loading Messages
When you work with translations typically you will need to load the translations before so that the application can use them. You can think of those files as static resources. You have a few options on how you deploy those files with the application:
Manually Setting The Translation Strings
NewPrinterBellow is an example application by loading translations on init:
1package main
2import (
3 "golang.org/x/text/language"
4 "golang.org/x/text/feature/plural"
5 "golang.org/x/text/message"
6 "golang.org/x/text/message/catalog"
7)
8type entry struct {
9 tag, key string
10 msg interface{}
11}
12var entries = [...]entry{
13 {"en", "Hello World", "Hello World"},
14 {"el", "Hello World", "Για Σου Κόσμε"},
15 {"en", "%d task(s) remaining!", plural.Selectf(1, "%d",
16 "=1", "One task remaining!",
17 "=2", "Two tasks remaining!",
18 "other", "[1]d tasks remaining!",
19 )},
20 {"el", "%d task(s) remaining!", plural.Selectf(1, "%d",
21 "=1", "Μία εργασία έμεινε!",
22 "=2", "Μια-δυο εργασίες έμειναν!",
23 "other", "[1]d εργασίες έμειναν!",
24 )},
25}
26func init() {
27 for _, e := range entries {
28 tag := language.MustParse(e.tag)
29 switch msg := e.msg.(type) {
30 case string:
31 message.SetString(tag, e.key, msg)
32 case catalog.Message:
33 message.Set(tag, e.key, msg)
34 case []catalog.Message:
35 message.Set(tag, e.key, msg...)
36 }
37 }
38}
39func main() {
40 p := message.NewPrinter(language.Greek)
41 p.Printf("Hello World")
42 p.Println()
43 p.Printf("%d task(s) remaining!", 2)
44 p.Println()
45 p = message.NewPrinter(language.English)
46 p.Printf("Hello World")
47 p.Println()
48 p.Printf("%d task(s) remaining!", 2)
49}If you run this program then it will print:
1$ go run examples/static/main.go
2Για Σου Κόσμε
3Μια-δυο εργασίες έμειναν!
4Hello World
5Two tasks remaining!%In practice, while this way is simple to implement, it’s not scalable enough. It works only for small applications with few translations. You will have to manually set the translation strings and it’s tricky to automate. For all other reasons, it’s recommended to automatically load messages where I explain in detail how to do it next.
Automatic Loading Of Messages
Traditionally, most localization frameworks have grouped data in per-language dynamically-loaded files. You can distribute those files to translators and have them merged into your app when they are ready.
gotextLet’s start by making sure that you have the latest version:
1$ go get -u golang.org/x/text/cmd/gotexthelp1$ gotext
2gotext is a tool for managing text in Go source code.
3Usage:
4 gotext command [arguments]
5The commands are:
6 update merge translations and generate catalog
7 extract extracts strings to be translated from code
8 rewrite rewrites fmt functions to use a message Printer
9 generate generates code to insert translated messagesFor the purposes of this tutorial let’s use the update flag which performs a multi-step process of extracting the translation keys to a file and updating the code for loading them into catalogs for ease of use.
main.goPrintFgo:generate1$ touch main.go- File: main.go
1package main
2//go:generate gotext -srclang=en update -out=catalog/catalog.go -lang=en,el
3import (
4 "golang.org/x/text/language"
5 "golang.org/x/text/message"
6)
7func main() {
8 p := message.NewPrinter(language.Greek)
9 p.Printf("Hello world!")
10 p.Println()
11 p.Printf("Hello", "world!")
12 p.Println()
13 person := "Alex"
14 place := "Utah"
15 p.Printf("Hello ", person, " in ", place, "!")
16 p.Println()
17 // Greet everyone.
18 p.Printf("Hello world!")
19 p.Println()
20 city := "Munich"
21 p.Printf("Hello %s!", city)
22 p.Println()
23 // Person visiting a place.
24 p.Printf("%s is visiting %s!",
25 person,
26 place)
27 p.Println()
28 // Double arguments.
29 miles := 1.2345
30 p.Printf("%.2[1]f miles traveled (%[1]f)", miles)
31}Run the following commands:
1$ mkdir catalog
2$ go generateThen fix the import to include the catalog.go file:
*File: main.go
1package main
2//go:generate gotext -srclang=en update -out=catalog/catalog.go -lang=en,el
3import (
4 "golang.org/x/text/language"
5 "golang.org/x/text/message"
6 _ "golang.org/x/text/message/catalog"
7)
8...Now if you see the project structure there are some files created:
1$ tree .
2.
3├── catalog
4│ └── catalog.go
5├── locales
6│ ├── el
7│ │ └── out.gotext.json
8│ └── en
9│ └── out.gotext.json
10├── main.gomessages.gotext.json1$ touch locales/el/messages.gotext.jsonFile: locales/el/messages.gotext.json
1{
2 "language": "el",
3 "messages": [
4 {
5 "id": "Hello world!",
6 "message": "Hello world!",
7 "translation": "Γιά σου Κόσμε!"
8 },
9 {
10 "id": "Hello",
11 "message": "Hello",
12 "translation": "Γιά σας %[1]v",
13 "placeholders": [
14 {
15 "id": "World",
16 "string": "%[1]v",
17 "type": "string",
18 "underlyingType": "string",
19 "argNum": 1,
20 "expr": "\"world!\""
21 }
22 ]
23 },
24 {
25 "id": "Hello {City}!",
26 "message": "Hello {City}!",
27 "translation": "Γιά σου %[1]s",
28 "placeholders": [
29 {
30 "id": "City",
31 "string": "%[1]s",
32 "type": "string",
33 "underlyingType": "string",
34 "argNum": 1,
35 "expr": "city"
36 }
37 ]
38 },
39 {
40 "id": "{Person} is visiting {Place}!",
41 "message": "{Person} is visiting {Place}!",
42 "translation": "Ο %[1]s επισκέπτεται την %[2]s",
43 "placeholders": [
44 {
45 "id": "Person",
46 "string": "%[1]s",
47 "type": "string",
48 "underlyingType": "string",
49 "argNum": 1,
50 "expr": "person"
51 },
52 {
53 "id": "Place",
54 "string": "%[2]s",
55 "type": "string",
56 "underlyingType": "string",
57 "argNum": 2,
58 "expr": "place"
59 }
60 ]
61 },
62 {
63 "id": "{Miles} miles traveled ({Miles_1})",
64 "message": "{Miles} miles traveled ({Miles_1})",
65 "translation": "%.2[1]f μίλια ταξίδεψε %[1]f",
66 "placeholders": [
67 {
68 "id": "Miles",
69 "string": "%.2[1]f",
70 "type": "float64",
71 "underlyingType": "float64",
72 "argNum": 1,
73 "expr": "miles"
74 },
75 {
76 "id": "Miles_1",
77 "string": "%[1]f",
78 "type": "float64",
79 "underlyingType": "float64",
80 "argNum": 1,
81 "expr": "miles"
82 }
83 ]
84 }
85 ]
86}go generate1$ go generate
2$ go run main.go
3Γιά σου Κόσμε!
4Γιά σας world!
5Γιά σου Κόσμε!
6Γιά σου Munich
7Ο Alex επισκέπτεται την Utah
81,23 μίλια ταξίδεψε 1,234500%rewritefmtp.PrintFile: main.go
1func main() {
2 p := message.NewPrinter(language.German)
3 fmt.Println("Hello world")
4 fmt.Printf("Hello world!")
5 p.Printf("Hello world!\n")
6}If you run the following command:
1$ gotext rewrite -out main.goThen the main.go will turn into:
1func main() {
2 p := message.NewPrinter(language.German)
3 p.Printf("Hello world\n")
4 p.Printf("Hello world!")
5 p.Printf("Hello world!\n")
6}Example Microservice
golang/x/textFirst, make sure you have all dependencies installed.
Start by creating an application skeleton:
1$ go get -u github.com/golang/dep/cmd/dep
2$ dep init
3$ touch main.goFile: main.go
1package main
2import (
3 "html"
4 "log"
5 "net/http"
6 "fmt"
7 "flag"
8 "time"
9)
10const (
11 httpPort = "8090"
12)
13func PrintMessage(w http.ResponseWriter, r *http.Request) {
14 fmt.Fprintf(w, "Hello, %s", html.EscapeString(r.Host))
15}
16func main() {
17 var port string
18 flag.StringVar(&port, "port", httpPort, "http port")
19 flag.Parse()
20 server := &http.Server{
21 Addr: ":" + port,
22 ReadTimeout: 10 * time.Second,
23 WriteTimeout: 10 * time.Second,
24 MaxHeaderBytes: 1 << 16,
25 Handler: http.HandlerFunc(PrintMessage)}
26 log.Fatal(server.ListenAndServe())
27}fmt.FprintFp.FprintF1func PrintMessage(w http.ResponseWriter, r *http.Request) {
2 p := message.NewPrinter(language.English)
3 p.Fprintf(w,"Hello, %v", html.EscapeString(r.Host))Add the following line to your source code and run the go generate command:
1//go:generate gotext -srclang=en update -out=catalog/catalog.go -lang=en,el1$ dep ensure -update
2$ go generate
3el: Missing entry for "Hello, {Host}".Provide translations for the missing entries:
1$ cp locales/el/out.gotext.json locales/el/messages.gotext.jsonFile: locales/el/messages.gotext.json
1{
2 "language": "el",
3 "messages": [
4 {
5 "id": "Hello, {Host}",
6 "message": "Hello, {Host}",
7 "translation": "Γιά σου %[1]v",
8 "placeholders": [
9 {
10 "id": "Host",
11 "string": "%[1]v",
12 "type": "string",
13 "underlyingType": "string",
14 "argNum": 1,
15 "expr": "html.EscapeString(r.Host)"
16 }
17 ]
18 }
19 ]
20}main.go1$ go generateFile: main.go
1package main
2import (
3 "html"
4 "log"
5 "net/http"
6 "flag"
7 "time"
8 "golang.org/x/text/message"
9 "golang.org/x/text/language"
10 _ "go-internationalization/catalog"
11)
12...message.DefaultCataloggotextFile: main.go
1var matcher = language.NewMatcher(message.DefaultCatalog.Languages())Add your function to match the correct language based on the request parameters:
File: main.go
1package main
2import (
3 "html"
4 "log"
5 "net/http"
6 "flag"
7 "time"
8 "context"
9 "golang.org/x/text/message"
10 "golang.org/x/text/language"
11 _ "go-internationalization/catalog"
12)
13//go:generate gotext -srclang=en update -out=catalog/catalog.go -lang=en,el
14var matcher = language.NewMatcher(message.DefaultCatalog.Languages())
15type contextKey int
16const (
17 httpPort = "8090"
18 messagePrinterKey contextKey = 1
19)
20func withMessagePrinter(next http.HandlerFunc) http.HandlerFunc {
21 return func(w http.ResponseWriter, r *http.Request) {
22 lang, ok := r.URL.Query()["lang"]
23 if !ok || len(lang) < 1 {
24 lang = append(lang, language.English.String())
25 }
26 tag,_, _ := matcher.Match(language.MustParse(lang[0]))
27 p := message.NewPrinter(tag)
28 ctx := context.WithValue(context.Background(), messagePrinterKey, p)
29 next.ServeHTTP(w, r.WithContext(ctx))
30 }
31}
32...I only supplied a parameter parsed from the query string. You can mix and match also additional tags parsed from a cookie or an Accept-Language header.
PrintMessagewithMessagePrinterFile: main.go
1...
2func PrintMessage(w http.ResponseWriter, r *http.Request) {
3 p := r.Context().Value(messagePrinterKey).(*message.Printer)
4 p.Fprintf(w,"Hello, %v", html.EscapeString(r.Host))
5}
6func main() {
7 var port string
8 flag.StringVar(&port, "port", httpPort, "http port")
9 flag.Parse()
10 server := &http.Server{
11 Addr: ":" + port,
12 ReadTimeout: 10 * time.Second,
13 WriteTimeout: 10 * time.Second,
14 MaxHeaderBytes: 1 << 16,
15 Handler: http.HandlerFunc(withMessagePrinter(PrintMessage))}
16 log.Fatal(server.ListenAndServe())
17}Start the server and issue some requests to see the translations happening:
1$ go run main.goThe world is your oyster from now on…
Use Phraseapp
PhraseApp supports many different languages and frameworks, including Go. It allows to easily import and export translations data and search for any missing translations, which is really convenient. On top of that, you can collaborate with translators as it is much better to have professionally done localization for your website. If you’d like to learn more about PhraseApp, refer to the Getting Started guide. You can also get a 14-days trial. So what are you waiting for?
Conclusion
golang/x/text packageThis is by no means an exhaustive guide as every application has different needs and scope requirements. Please stay put for more detailed articles regarding this subject.