I am fetching JSON files in go that are not formatted homogeneously. For Example, I can have the following:

{"email": "\"blah.blah@blah.com\""}
{"email": "robert@gmail.com"}
{"name": "m\303\203ead"}

json.Decode

With:

{"name": "m\303\203ead"}

invalid character '3' in string escape code

I have tried several approaches to normalise my data for example by passing by a string array (it works but there is too many edge cases), or even to filter escape characters.

Finally, I came through this article: (http://blog.golang.org/normalization) And the solution they proposed seemed very interesting.

I have tried the following

isMn := func(r rune) bool {
    return unicode.Is(unicode.Mn, r)
}

t := transform.Chain(norm.NFC, transform.RemoveFunc(isMn), norm.NFD)

fileReader, err := bucket.GetReader(filename)

transformReader := transform.NewReader(fileReader, t)

decoder := json.NewDecoder(tReader)

for {
    var dataModel Model
    if err := decoder.Decode(&kmData); err == io.EOF {
        break
    } else {
      // DO SOMETHING
    }
}

Model
type Model struct {
    Name  string `json:"name" bson:"name"`
    Email string `json:"email" bson:"email"` 
}

I have tried several variations of it, but haven't been able to have it working.

So my question is how to easily handle decoding/unmarshaling JSON data with different encodings? Knowing, that I have no control on those JSON files.

If you are reading this, thank you anyway.