jsonmanu is a Go library intended to be used as a JSON data manipulator by retrieving and updating JSON object sub-branches as well as mapping data between two JSON objects. The data querying is done by utilizing the JSONPath notation.
- JSON MANipUlator - Table of contents
You can install the library using the go toolchain:
go get -u -v github.com/antavelos/jsonmanugo test -vThe main api of the library consists of:
- Get which is used to retrieve specific branches/leafs of JSON data as described by the provided JSONPath
- Put which is used to update specific branches/leafs of JSON data as described by the provided JSONPath.
- Map which is used to map data from a JSON data source to a JSON data destination.
In the following sections describe the library api in details along with examples. For more information about the supported JSPNPath variations please refer below.
It accepts:
dataof typemap[string]anythe typical type of an unmarshalled JSON object.pathwhich must be a string complying with the JSONPath usecases.
It returns:
- a value of type
anyas it can be any of the types handled by json.Unmarshal function. - an error in case something went wrong either during the path parsing or the value retrieval.
Here is an example:
package main
import (
"encoding/json"
"fmt"
jm "github.com/antavelos/jsonmanu"
)
var jsonStr = []byte(`{
"store": {
"library": {
"books": [
{
"author": "Nietzsche",
"title": "Book1",
"price": 15
},
{
"author": "Schopenhauer",
"title": "Book2",
"price": 20
},
{
"author": "Stirner",
"title": "Book3",
"price": 15
},
{
"author": "Camus",
"title": "Book4",
"price": 5
},
{
"author": "Dostoevsky",
"title": "Book5",
"price": 10
},
{
"author": "Heraklitus",
"title": "Book6",
"price": 10
}
]
}
}
}`)
func main() {
var data any
err := json.Unmarshal(jsonStr, &data)
if err != nil {
panic(err)
}
// get all authors
fmt.Println(jm.Get(data, "$.store..author"))
// ["Nietzsche" "Schopenhauer" "Stirner" "Camus" "Dostoevsky" "Heraklitus"] <nil>
// get the price of 1st and 6th books
fmt.Println(jm.Get(data, "$..books[0,5].price"))
// [15 10] <nil>
// get the title of 2nd and 5th books
fmt.Println(jm.Get(data, "$..books[1:4].title"))
// ["Book2" "Book3" "Book4"] <nil>
// get the price of books authored by Nietzsche
fmt.Println(jm.Get(data, "$..books[?(@.author == Nietzsche)].price"))
// [15] <nil>
}It accepts:
dataof typemap[string]anythe typical type of an unmarshalled JSON object.pathof typestringcomplying with the JSONPath usecases.valueof typeanyas it can be any of the types handled by json.Unmarshal function.
It returns:
- an error in case something went wrong either during the path parsing or the data update.
After the call the data variable will contain the update version of it.
Here is an example:
package main
import (
"encoding/json"
"fmt"
jm "github.com/antavelos/jsonmanu"
)
var jsonStr = []byte(`{
"store": {
"library": {
"books": [
{
"author": "Nietzsche",
"title": "Book1",
"price": 15
},
{
"author": "Schopenhauer",
"title": "Book2",
"price": 20
},
{
"author": "Stirner",
"title": "Book3",
"price": 15
},
{
"author": "Camus",
"title": "Book4",
"price": 5
},
{
"author": "Dostoevsky",
"title": "Book5",
"price": 10
},
{
"author": "Heraklitus",
"title": "Book6",
"price": 10
}
]
}
}
}`)
func main() {
var data any
err := json.Unmarshal(jsonStr, &data)
if err != nil {
panic(err)
}
// update the author of the 2nd book to Nietzsche
jm.Put(data, "$..books[1].author", "Nietzsche")
fmt.Println(data["store"]["library"]["books"][1])
// map[author:Nietzsche price:20 title:Book2]
// update the price of books authored by Camus to 30
jm.Put(data, "$..books[?(@.author == Camus)].price", 30)
fmt.Println(data["store"]["library"]["books"][3])
// map[author:Camus price:30 title:Book4]
}It accepts:
srcof typemap[string]anywhich is the source object to be mapped.dstof typemap[string]anywhich is the destination object where source will be mapped to. It cannot be nil.mappersof typeMapperwhich is a list of JSONPath pairs (source and destination) along with one or more optional transformations that will apply (in chain mode and in the order of cofiguration) on the retrieved source value before its put in the destionation object.
It returns:
- a list of errors per mapper.
After the call, the changes in dst apply in place.
Here is an example:
package main
import (
"encoding/json"
"fmt"
jm "github.com/antavelos/jsonmanu"
)
var sourceJsonStr = []byte(`{
"store": {
"library": {
"books": [
{
"author": "Nietzsche",
"title": "Book1",
"price": 15,
"summary": "born on 15/10/1844"
},
{
"author": "Schopenhauer",
"title": "Book2",
"price": 20,
"summary": "born on 22/02/1788"
},
{
"author": "Stirner",
"title": "Book3",
"price": 15,
"summary": "born on 25/10/1806"
},
{
"author": "Camus",
"title": "Book4",
"price": 5,
"summary": "born on 07/11/1913"
},
{
"author": "Dostoevsky",
"title": "Book5",
"price": 10,
"summary": "born on 30/10/1821"
}
]
}
}
}`)
func main() {
var sourceData map[string]any
destData := make(map[string]any)
err := json.Unmarshal(sourceJsonStr, &sourceData)
if err != nil {
panic(err)
}
mappers := []jm.Mapper{
jm.Mapper{
SrcJsonPath: "$.store.library.books.author",
DstJsonPath: "$.info.authors",
},
jm.Mapper{
SrcJsonPath: "$.store.library.books.summary",
DstJsonPath: "$.info.birthYears",
Transformations: []jm.Transformation{
{Trsnfmr: jm.StringMatchTransformer{Regex: `\d{2}/\d{2}/\d{4}`}},
{Trsnfmr: jm.SplitTransformer{Delim: "/", Index: 2}},
{Trsnfmr: jm.NumberTransformer{}},
},
},
}
jm.Map(sourceData, destData, mappers)
prettyDstData, _ := json.MarshalIndent(destData, "", " ")
fmt.Printf("%s", prettyDstData)
// {
// "info": {
// "authors": [
// "Nietzsche",
// "Schopenhauer",
// "Stirner",
// "Camus",
// "Dostoevsky"
// ],
// "birthYears": [
// 1844,
// 1788,
// 1806,
// 1913,
// 1821
// ]
// }
// }
}
Transformation is a struct type that contains:
- a Transformer and
- an
asArrayflag: Since a retrieved source value can often be an array, this flag can be used to indicate whether the transformation will apply on the source value as an array or it will apply on each item individualy.
Transformer is a interface that implements the Transform(value any) (any, error) method. That means that it's up to the user to define their custom transformers in order to manipulate the retrieved data.
However, there are already some pre-built transformers that can be mainly used for string manipulation purposes:
type SplitTransformer struct {
Delim string
Index int
}SplitTransformer will split a string value based on the provided delimeter and it will pick the element defined by the provided index from the occured array.
type JoinTransformer struct {
Delim string
}JoinTransformer joins the values of an array based on the provided delimiter.
type ReplaceTransformer struct {
OldVal string
NewVal string
}ReplaceTransformer replaces a substring in a string value with another.
type StringMatchTransformer struct {
Regex string
}StringMatchTransformer returns a regex matched substring of a string.
type SubStrTransformer struct {
Start int
End int
}SubStrTransformer returns a slice of a string value based on the provided indices.
type NumberTransformer struct{}NumberTransformer converts a string value to float64.
Here is the complete list of the JSONPath supported (or not yet) usecases:
| Expression | Description | Supported |
|---|---|---|
| $ | The root object of array | YES |
| .property | Selects the specified property in a parent object. | YES |
| ['property'] | Selects the specified property in a parent object. | NO |
| [n] | Selects the n-th element from an array. Indexes are 0-based. | YES |
| [index1,index2,...] | Selects array elements with the specified indexes. Returns a list. | YES |
| ..property | Recursive descent: Searches for the specified property name recursively and returns an array of all values with this property name. Always returns a list, even if just one property is found. | YES |
| * | Wildcard selects all elements in an object or an array, regardless of their names or indexes. For example, address.* means all properties of the address object, and book[*] means all items of the book array. | YES |
| [start:end] [start:] | Selects array elements from the start index and up to, but not including, end index. If end is omitted, selects all elements from start until the end of the array. Returns a list. | YES |
| [:n] | Selects the first n elements of the array. Returns a list. | YES |
| [-n:] | Selects the last n elements of the array. Returns a list. | NO |
| [?(expression)] | Filter expression. Selects all elements in an object or array that match the specified filter. Returns a list. You can see mre details in the below section. | YES |
| @ | Used in filter expressions to refer to the current node being processed. | YES |
With an expression you can filter array elements bases of the properties of its object items. The supported operators are ==, !=, <, >, <=, >= and they apply on both numbers an strings.
Examples:
$.books[?(author == "Nietzsche")]filters all the book authored by Nietzsche.$.books[?(author != "Nietzsche")]filters all the books except from those authored by Nietzsche.$.books[?(price == 10)]filters all the books with price equal to 10.$.books[?(price != 10)]filters all the books with price not equal to 10.$.books[?(price >= 10)]filters all the books with price greater or equal to 10.$.books[?(price <= 10)]filters all the books with price less or equal to 10.$.books[?(price > 10)]filters all the books with price greater than 10.$.books[?(price < 10)]filters all the books with price less than 10.
See LICENSE file.