我最近看到go yaml库具有新版本(V3)
具有nodes功能(在我看来,这是一个杀手级功能:)),该功能可以在不更改文件结构的情况下帮助大量修改yamls
但是,由于它是相当新的(从上周开始),所以我没有找到一些有用的文档和示例,这些文档和示例只适合我需要的上下文(添加新的对象/节点并保留文件结构相同,而不会删除注释等)
我需要的是操作yaml文件
例如
让我说我有这个Yaml文件
version: 1
type: verbose
kind : bfr
# my list of applications
applications:
- name: app1
kind: nodejs
path: app1
exec:
platforms: k8s
builder: test
现在我有一个json对象(例如,app2
),我需要将其插入到现有文件中
[
{
"comment: "Second app",
"name": "app2",
"kind": "golang",
"path": "app2",
"exec": {
"platforms": "dockerh",
"builder": "test"
}
}
]
第一个应用程序之后,我需要将其添加到yml文件中(应用程序是应用程序的数组)
version: 1
type: verbose
kind : bfr
# my list of applications
applications:
# First app
- name: app1
kind: nodejs
path: app1
exec:
platforms: k8s
builder: test
# Second app
- name: app2
kind: golang
path: app2
exec:
platforms: dockerh
builder: test
是否可以从yaml文件中添加新的json对象?还删除现有的
我也找到了此博客 https://blog.ubuntu.com/2019/04/05/api-v3-of-the-yaml-package-for-go-is-available
这是代表对象的类型
type VTS struct {
version string `yaml:"version"`
types string `yaml:"type"`
kind string `yaml:"kind,omitempty"`
apps Applications `yaml:"applications,omitempty"`
}
type Applications []struct {
Name string `yaml:"name,omitempty"`
Kind string `yaml:"kind,omitempty"`
Path string `yaml:"path,omitempty"`
Exec struct {
Platforms string `yaml:"platforms,omitempty"`
Builder string `yaml:"builder,omitempty"`
} `yaml:"exec,omitempty"`
}
更新
在测试wiil7200
提供的解决方案之后,我发现了2个问题
我最后使用它写入文件
err = ioutil.WriteFile("output.yaml", b, 0644)
并且yaml输出有2个问题。
应用程序的数组从注释开始,应该 从名字开始
- 不对齐
在
name
条目之后,kind
属性和之后的所有其他属性 与name
有什么办法解决那些问题吗?关于comments
问题,可以说我是从其他财产那里得到的
而不是来自json(如果它使它更简单)
version: 1
type: verbose
kind: bfr
# my list of applications
applications:
- # First app
name: app1
kind: nodejs
path: app1
exec:
platforms: k8s
builder: test
- # test 1
name: app2
kind: golang
path: app2
exec:
platform: dockerh
builder: test
答案 0 :(得分:6)
首先,让我开始说使用yaml。当从有效yaml编组时,Node不会产生有效的yaml,如以下示例所示。可能应该提出问题。
package main
import (
"fmt"
"log"
"gopkg.in/yaml.v3"
)
var (
sourceYaml = `version: 1
type: verbose
kind : bfr
# my list of applications
applications:
# First app
- name: app1
kind: nodejs
path: app1
exec:
platforms: k8s
builder: test
`
)
func main() {
t := yaml.Node{}
err := yaml.Unmarshal([]byte(sourceYaml), &t)
if err != nil {
log.Fatalf("error: %v", err)
}
b, err := yaml.Marshal(&t)
if err != nil {
log.Fatal(err)
}
fmt.Println(string(b))
}
在go版本go1.12.3 windows / amd64中产生以下无效的Yaml
version: 1
type: verbose
kind: bfr
# my list of applications
applications:
- # First app
name: app1
kind: nodejs
path: app1
exec:
platforms: k8s
builder: test
其次,使用
之类的结构type VTS struct {
Version string `yaml:"version" json:"version"`
Types string `yaml:"type" json:"type"`
Kind string `yaml:"kind,omitempty" json:"kind,omitempty"`
Apps yaml.Node `yaml:"applications,omitempty" json:"applications,omitempty"`
}
从ubuntu的博客和源文档中可以看出,它似乎可以正确识别结构中作为节点的字段,并分别构建该树,但事实并非如此。 取消编组时,它将提供正确的节点树,但是重新编组时,它将产生以下yaml,其中包含yaml.Node公开的所有字段。可悲的是,我们不能走这条路,必须找到另一条路。
version: "1"
type: verbose
kind: bfr
applications:
kind: 2
style: 0
tag: '!!seq'
value: ""
anchor: ""
alias: null
content:
- # First app
name: app1
kind: nodejs
path: app1
exec:
platforms: k8s
builder: test
headcomment: ""
linecomment: ""
footcomment: ""
line: 9
column: 3
我们忽略了结构中yaml.Nodes的第一个问题和程序错误(位于gopkg.in/yaml.v3 v3.0.0-20190409140830-cdc409dda467上),我们现在可以开始处理程序包公开的Nodes。不幸的是,没有可以轻松添加节点的抽象,因此用途可能会有所不同,并且标识节点可能会很麻烦。反思可能会对您有所帮助,因此我将其作为练习留给您。
您会发现注释spew.dump以一种不错的格式转储了整个节点树,这有助于在将Nodes添加到源树中时进行调试。
您当然也可以删除节点,您只需要确定需要删除的特定节点即可。您只需确保删除父节点(如果它们是图或序列)即可。
package main
import (
"encoding/json"
"fmt"
"log"
"gopkg.in/yaml.v3"
)
var (
sourceYaml = `version: 1
type: verbose
kind : bfr
# my list of applications
applications:
# First app
- name: app1
kind: nodejs
path: app1
exec:
platforms: k8s
builder: test
`
modifyJsonSource = `
[
{
"comment": "Second app",
"name": "app2",
"kind": "golang",
"path": "app2",
"exec": {
"platforms": "dockerh",
"builder": "test"
}
}
]
`
)
// VTS Need to Make Fields Public otherwise unmarshalling will not fill in the unexported fields.
type VTS struct {
Version string `yaml:"version" json:"version"`
Types string `yaml:"type" json:"type"`
Kind string `yaml:"kind,omitempty" json:"kind,omitempty"`
Apps Applications `yaml:"applications,omitempty" json:"applications,omitempty"`
}
type Applications []struct {
Name string `yaml:"name,omitempty" json:"name,omitempty"`
Kind string `yaml:"kind,omitempty" json:"kind,omitempty"`
Path string `yaml:"path,omitempty" json:"path,omitempty"`
Exec struct {
Platforms string `yaml:"platforms,omitempty" json:"platforms,omitempty"`
Builder string `yaml:"builder,omitempty" json:"builder,omitempty"`
} `yaml:"exec,omitempty" json:"exec,omitempty"`
Comment string `yaml:"comment,omitempty" json:"comment,omitempty"`
}
func main() {
t := yaml.Node{}
err := yaml.Unmarshal([]byte(sourceYaml), &t)
if err != nil {
log.Fatalf("error: %v", err)
}
// Look for the Map Node with the seq array of items
applicationNode := iterateNode(&t, "applications")
// spew.Dump(iterateNode(&t, "applications"))
var addFromJson Applications
err = json.Unmarshal([]byte(modifyJsonSource), &addFromJson)
if err != nil {
log.Fatalf("error: %v", err)
}
// Delete the Original Applications the following options:
// applicationNode.Content = []*yaml.Node{}
// deleteAllContents(applicationNode)
deleteApplication(applicationNode, "name", "app1")
for _, app := range addFromJson {
// Build New Map Node for new sequences coming in from json
mapNode := &yaml.Node{Kind: yaml.MappingNode, Tag: "!!map"}
// Build Name, Kind, and Path Nodes
mapNode.Content = append(mapNode.Content, buildStringNodes("name", app.Name, app.Comment)...)
mapNode.Content = append(mapNode.Content, buildStringNodes("kind", app.Kind, "")...)
mapNode.Content = append(mapNode.Content, buildStringNodes("path", app.Path, "")...)
// Build the Exec Nodes and the Platform and Builder Nodes within it
keyMapNode, keyMapValuesNode := buildMapNodes("exec")
keyMapValuesNode.Content = append(keyMapValuesNode.Content, buildStringNodes("platform", app.Exec.Platforms, "")...)
keyMapValuesNode.Content = append(keyMapValuesNode.Content, buildStringNodes("builder", app.Exec.Builder, "")...)
// Add to parent map Node
mapNode.Content = append(mapNode.Content, keyMapNode, keyMapValuesNode)
// Add to applications Node
applicationNode.Content = append(applicationNode.Content, mapNode)
}
// spew.Dump(t)
b, err := yaml.Marshal(&t)
if err != nil {
log.Fatal(err)
}
fmt.Println(string(b))
}
// iterateNode will recursive look for the node following the identifier Node,
// as go-yaml has a node for the key and the value itself
// we want to manipulate the value Node
func iterateNode(node *yaml.Node, identifier string) *yaml.Node {
returnNode := false
for _, n := range node.Content {
if n.Value == identifier {
returnNode = true
continue
}
if returnNode {
return n
}
if len(n.Content) > 0 {
ac_node := iterateNode(n, identifier)
if ac_node != nil {
return ac_node
}
}
}
return nil
}
// deleteAllContents will remove all the contents of a node
// Mark sure to pass the correct node in otherwise bad things will happen
func deleteAllContents(node *yaml.Node) {
node.Content = []*yaml.Node{}
}
// deleteApplication expects that a sequence Node with all the applications are present
// if the key value are not found it will not log any errors, and return silently
// this is expecting a map like structure for the applications
func deleteApplication(node *yaml.Node, key, value string) {
state := -1
indexRemove := -1
for index, parentNode := range node.Content {
for _, childNode := range parentNode.Content {
if key == childNode.Value && state == -1 {
state += 1
continue // found expected move onto next
}
if value == childNode.Value && state == 0 {
state += 1
indexRemove = index
break // found the target exit out of the loop
} else if state == 0 {
state = -1
}
}
}
if state == 1 {
// Remove node from contents
// node.Content = append(node.Content[:indexRemove], node.Content[indexRemove+1:]...)
// Don't Do this you might have a potential memory leak source: https://github.com/golang/go/wiki/SliceTricks
// Since the underlying nodes are pointers
length := len(node.Content)
copy(node.Content[indexRemove:], node.Content[indexRemove+1:])
node.Content[length-1] = nil
node.Content = node.Content[:length-1]
}
}
// buildStringNodes builds Nodes for a single key: value instance
func buildStringNodes(key, value, comment string) []*yaml.Node {
keyNode := &yaml.Node{
Kind: yaml.ScalarNode,
Tag: "!!str",
Value: key,
HeadComment: comment,
}
valueNode := &yaml.Node{
Kind: yaml.ScalarNode,
Tag: "!!str",
Value: value,
}
return []*yaml.Node{keyNode, valueNode}
}
// buildMapNodes builds Nodes for a key: map instance
func buildMapNodes(key string) (*yaml.Node, *yaml.Node) {
n1, n2 := &yaml.Node{
Kind: yaml.ScalarNode,
Tag: "!!str",
Value: key,
}, &yaml.Node{Kind: yaml.MappingNode,
Tag: "!!map",
}
return n1, n2
}
生产Yaml
version: 1
type: verbose
kind: bfr
# my list of applications
applications:
- # First app
name: app1
kind: nodejs
path: app1
exec:
platforms: k8s
builder: test
- # Second app
name: app2
kind: golang
path: app2
exec:
platform: dockerh
builder: test
答案 1 :(得分:0)
您可以创建一个新节点并直接附加到内容,而无需删除前一个节点。下面的示例说明了这一点:
package main
import (
"fmt"
"log"
"gopkg.in/yaml.v3"
)
var (
sourceYaml = `version: 1
type: verbose
kind : bfr
# my list of applications
applications:
# First app
- name: app1
kind: nodejs
path: app1
exec:
platforms: k8s
builder: test
`
)
type Application struct {
Name string `yaml:"name,omitempty" json:"name,omitempty"`
Kind string `yaml:"kind,omitempty" json:"kind,omitempty"`
Path string `yaml:"path,omitempty" json:"path,omitempty"`
Exec struct {
Platforms string `yaml:"platforms,omitempty" json:"platforms,omitempty"`
Builder string `yaml:"builder,omitempty" json:"builder,omitempty"`
} `yaml:"exec,omitempty" json:"exec,omitempty"`
}
func newApplicationNode(
name string,
kind string,
path string,
platforms string,
builder string,
comment string) (*yaml.Node, error) {
app := Application{
Name: name,
Kind: kind,
Path: path,
Exec: struct {
Platforms string `yaml:"platforms,omitempty" json:"platforms,omitempty"`
Builder string `yaml:"builder,omitempty" json:"builder,omitempty"`
}{platforms, builder},
}
marshalledApp, err := yaml.Marshal(&app)
if err != nil {
return nil, err
}
node := yaml.Node{}
if err := yaml.Unmarshal(marshalledApp, &node); err != nil {
return nil, err
}
node.Content[0].HeadComment = comment
return &node, nil
}
func main() {
yamlNode := yaml.Node{}
err := yaml.Unmarshal([]byte(sourceYaml), &yamlNode)
if err != nil {
log.Fatalf("error: %v", err)
}
newApp, err := newApplicationNode("app2", "golang", "app2", "dockerh",
"test", "Second app")
if err != nil {
log.Fatalf("error: %v", err)
}
appIdx := -1
for i, k := range yamlNode.Content[0].Content {
if k.Value == "applications" {
appIdx = i + 1
break
}
}
yamlNode.Content[0].Content[appIdx].Content = append(
yamlNode.Content[0].Content[appIdx].Content, newApp.Content[0])
out, err := yaml.Marshal(&yamlNode)
if err != nil {
log.Fatal(err)
}
fmt.Println(string(out))
}
很明显,您可以像从newApplicationNode
中那样采用骇人听闻的方式,而可以从JSON中正确解组。但是,如先前的回答所述,重要的是要注意键和实际值位于Content
内的后续索引中,因此在修改文档时需要考虑到这一点。 (例如,查找applications
键,然后考虑其内容的下一个索引(在我的示例中为appIdx = i + 1
)。
希望有帮助!