Skip to content

Commit 937f0eb

Browse files
schmichaeltgross
schmichael
and
tgross
committed
docs: update json jobs docs (#12766)
* docs: update json jobs docs Did you know that Nomad has not 1 but 2 JSON formats for jobs? 2½ if you want to acknowledge that sometimes our JSON job representations have a Job top-level wrapper and sometimes do not. The 2½ formats are: ``` 1. HCL JSON 2. Input API JSON (top-level Job field) 2.5. Output API JSON (lacks top-level Job field) ``` `#2` is what our docs consider our API JSON. `#2.5` seems to be an accident of history we can't fix with breaking API compatibility. `#1` is an even more interesting accident of history: the `jobspec2` package automatically detects if the input to Parse is JSON and switches to a JSON parser. This behavior is undocumented, the format is unspecified, and there is no official HashiCorp tooling to produce this JSON from HCL. The plot thickens when you discover popular third party tools like hcl2json.com and https://github.com/tmccombs/hcl2json seem to produce JSON that `nomad run` accepts! Since we have no telemetry around whether or not anyone passes HCL JSON to `nomad run`, and people don't file bugs around features that Just Work, I'm choosing to leave that code path in place and *acknowledged but not suggested* in documentation. See hashicorp/hcl#498 for a more comprehensive discussion of what officially supporting HCL JSON in Nomad would look like. (I also added some of the missing fields to the (Input API flavor) JSON Job documentation, but it still needs a lot of work to be comprehensive.) Co-authored-by: Tim Gross <[email protected]>
1 parent a0374cd commit 937f0eb

File tree

3 files changed

+220
-106
lines changed

3 files changed

+220
-106
lines changed

‎website/content/api-docs/json-jobs.mdx

+184-91
Original file line numberDiff line numberDiff line change
@@ -8,149 +8,221 @@ description: |-
88

99
# JSON Job Specification
1010

11-
This guide covers the JSON syntax for submitting jobs to Nomad. A useful command
12-
for generating valid JSON versions of HCL jobs is:
11+
Nomad's HTTP API uses JSON formatted job specifications except for the
12+
[`/job/parse` API][job-parse] which exists to convert HCL to JSON.
13+
14+
The Nomad CLI includes a number of useful commands for working with JSON jobs.
15+
16+
The [`nomad job run -output`][job-output] flag converts HCL jobs to JSON
17+
without submitting the job:
1318

1419
```shell-session
1520
$ nomad job run -output my-job.nomad
1621
```
1722

23+
The [`nomad job inspect`][job-inspect] command retrieves the JSON specification
24+
for an existing job:
25+
26+
```shell-session
27+
$ nomad job inspect example
28+
```
29+
30+
The [`nomad job run -json`][job-run-json] flag submits a JSON formatted job:
31+
32+
```shell-session
33+
$ nomad job run -json example.json
34+
```
35+
36+
[job-inspect]: /docs/commands/job/inspect
37+
[job-output]: /docs/commands/job/run#output
38+
[job-parse]: /api-docs/jobs#parse-job
39+
[job-run-json]: /docs/commands/job/run#json
40+
1841
## Syntax
1942

20-
Below is the JSON representation of the job outputted by `$ nomad init`:
43+
Below is the JSON representation of the example job as well as the commands to
44+
reproduce it:
45+
46+
```shell-session
47+
$ nomad init
48+
Example job file written to example.nomad
49+
$ nomad job run -output example.nomad
50+
```
2151

2252
```json
2353
{
2454
"Job": {
55+
"Region": null,
56+
"Namespace": null,
2557
"ID": "example",
2658
"Name": "example",
2759
"Type": "service",
28-
"Priority": 50,
29-
"Datacenters": ["dc1"],
60+
"Priority": null,
61+
"AllAtOnce": null,
62+
"Datacenters": [
63+
"dc1"
64+
],
65+
"Constraints": null,
66+
"Affinities": null,
3067
"TaskGroups": [
3168
{
3269
"Name": "cache",
3370
"Count": 1,
34-
"Migrate": {
35-
"HealthCheck": "checks",
36-
"HealthyDeadline": 300000000000,
37-
"MaxParallel": 1,
38-
"MinHealthyTime": 10000000000
39-
},
71+
"Constraints": null,
72+
"Affinities": null,
4073
"Tasks": [
4174
{
4275
"Name": "redis",
4376
"Driver": "docker",
4477
"User": "",
78+
"Lifecycle": null,
4579
"Config": {
4680
"image": "redis:3.2",
47-
"port_map": [
48-
{
49-
"db": 6379
50-
}
81+
"ports": [
82+
"db"
5183
]
5284
},
53-
"Services": [
54-
{
55-
"Id": "",
56-
"Name": "redis-cache",
57-
"Tags": ["global", "cache"],
58-
"Meta": {
59-
"meta": "for my service"
60-
},
61-
"PortLabel": "db",
62-
"AddressMode": "",
63-
"Checks": [
64-
{
65-
"Id": "",
66-
"Name": "alive",
67-
"Type": "tcp",
68-
"Command": "",
69-
"Args": null,
70-
"Header": {},
71-
"Method": "",
72-
"Path": "",
73-
"Protocol": "",
74-
"PortLabel": "",
75-
"Interval": 10000000000,
76-
"Timeout": 2000000000,
77-
"InitialStatus": "",
78-
"TLSSkipVerify": false,
79-
"CheckRestart": {
80-
"Limit": 3,
81-
"Grace": 30000000000,
82-
"IgnoreWarnings": false
83-
}
84-
}
85-
]
86-
}
87-
],
85+
"Constraints": null,
86+
"Affinities": null,
87+
"Env": null,
88+
"Services": null,
8889
"Resources": {
8990
"CPU": 500,
91+
"Cores": null,
9092
"MemoryMB": 256,
91-
"Networks": [
92-
{
93-
"Device": "",
94-
"CIDR": "",
95-
"IP": "",
96-
"MBits": 10,
97-
"DynamicPorts": [
98-
{
99-
"Label": "db",
100-
"Value": 0
101-
}
102-
]
103-
}
104-
]
93+
"MemoryMaxMB": null,
94+
"DiskMB": null,
95+
"Networks": null,
96+
"Devices": null,
97+
"IOPS": null
10598
},
106-
"Leader": false
99+
"RestartPolicy": null,
100+
"Meta": null,
101+
"KillTimeout": null,
102+
"LogConfig": null,
103+
"Artifacts": null,
104+
"Vault": null,
105+
"Templates": null,
106+
"DispatchPayload": null,
107+
"VolumeMounts": null,
108+
"Leader": false,
109+
"ShutdownDelay": 0,
110+
"KillSignal": "",
111+
"Kind": "",
112+
"ScalingPolicies": null
107113
}
108114
],
115+
"Spreads": null,
116+
"Volumes": null,
109117
"RestartPolicy": {
110118
"Interval": 1800000000000,
111119
"Attempts": 2,
112120
"Delay": 15000000000,
113121
"Mode": "fail"
114122
},
115-
"ReschedulePolicy": {
116-
"Attempts": 10,
117-
"Delay": 30000000000,
118-
"DelayFunction": "exponential",
119-
"Interval": 0,
120-
"MaxDelay": 3600000000000,
121-
"Unlimited": true
122-
},
123+
"ReschedulePolicy": null,
123124
"EphemeralDisk": {
125+
"Sticky": null,
126+
"Migrate": null,
124127
"SizeMB": 300
125-
}
128+
},
129+
"Update": null,
130+
"Migrate": null,
131+
"Networks": [
132+
{
133+
"Mode": "",
134+
"Device": "",
135+
"CIDR": "",
136+
"IP": "",
137+
"DNS": null,
138+
"ReservedPorts": null,
139+
"DynamicPorts": [
140+
{
141+
"Label": "db",
142+
"Value": 0,
143+
"To": 6379,
144+
"HostNetwork": ""
145+
}
146+
],
147+
"Hostname": "",
148+
"MBits": null
149+
}
150+
],
151+
"Meta": null,
152+
"Services": [
153+
{
154+
"Id": "",
155+
"Name": "redis-cache",
156+
"Tags": [
157+
"global",
158+
"cache"
159+
],
160+
"CanaryTags": null,
161+
"EnableTagOverride": false,
162+
"PortLabel": "db",
163+
"AddressMode": "",
164+
"Checks": null,
165+
"CheckRestart": null,
166+
"Connect": null,
167+
"Meta": null,
168+
"CanaryMeta": null,
169+
"TaskName": "",
170+
"OnUpdate": "",
171+
"Provider": ""
172+
}
173+
],
174+
"ShutdownDelay": null,
175+
"StopAfterClientDisconnect": null,
176+
"MaxClientDisconnect": null,
177+
"Scaling": null,
178+
"Consul": null
126179
}
127180
],
128181
"Update": {
182+
"Stagger": null,
129183
"MaxParallel": 1,
184+
"HealthCheck": null,
130185
"MinHealthyTime": 10000000000,
131186
"HealthyDeadline": 180000000000,
187+
"ProgressDeadline": 600000000000,
188+
"Canary": 0,
132189
"AutoRevert": false,
133-
"Canary": 0
134-
}
190+
"AutoPromote": null
191+
},
192+
"Multiregion": null,
193+
"Spreads": null,
194+
"Periodic": null,
195+
"ParameterizedJob": null,
196+
"Reschedule": null,
197+
"Migrate": {
198+
"MaxParallel": 1,
199+
"HealthCheck": "checks",
200+
"MinHealthyTime": 10000000000,
201+
"HealthyDeadline": 300000000000
202+
},
203+
"Meta": null,
204+
"ConsulToken": null,
205+
"VaultToken": null,
206+
"Stop": null,
207+
"ParentID": null,
208+
"Dispatched": false,
209+
"DispatchIdempotencyToken": null,
210+
"Payload": null,
211+
"ConsulNamespace": null,
212+
"VaultNamespace": null,
213+
"NomadTokenID": null,
214+
"Status": null,
215+
"StatusDescription": null,
216+
"Stable": null,
217+
"Version": null,
218+
"SubmitTime": null,
219+
"CreateIndex": null,
220+
"ModifyIndex": null,
221+
"JobModifyIndex": null
135222
}
136223
}
137224
```
138225

139-
The example JSON could be submitted as a job using the following:
140-
141-
```shell-session
142-
$ curl -XPUT -d @example.json http://127.0.0.1:4646/v1/job/example
143-
{
144-
"EvalID": "5d6ded54-0b2a-8858-6583-be5f476dec9d",
145-
"EvalCreateIndex": 12,
146-
"JobModifyIndex": 11,
147-
"Warnings": "",
148-
"Index": 12,
149-
"LastContact": 0,
150-
"KnownLeader": false
151-
}
152-
```
153-
154226
## Syntax Reference
155227

156228
Following is a syntax reference for the possible keys that are supported and
@@ -183,6 +255,15 @@ The `Job` object supports the following keys:
183255

184256
- `Meta` - Annotates the job with opaque metadata.
185257

258+
- `ConsulToken` - Specifies the Consul token that proves the submitter of the
259+
job has access to the Service Identity policies associated with the job's
260+
Consul Connect enabled services. This field is only used to transfer the
261+
token and is not stored after job submission.
262+
263+
- `VaultToken` - Specifies the Vault token that proves the submitter of the job
264+
has access to the specified policies in the `vault` stanza. This field is
265+
only used to transfer the token and is not stored after job submission.
266+
186267
- `Namespace` - The namespace to execute the job in, defaults to "default".
187268
Prior to Nomad 1.0 namespaces were Enterprise-only.
188269

@@ -200,6 +281,9 @@ The `Job` object supports the following keys:
200281
dispatching against the parameterized job. The options for this field are
201282
"optional", "required" and "forbidden". The default value is "optional".
202283

284+
- `DispatchIdempotencyToken` - Optional identifier used to prevent more than one
285+
instance of the job from being dispatched.
286+
203287
- `Payload` - The payload may not be set when submitting a job but may appear in
204288
a dispatched job. The `Payload` will be a base64 encoded string containing the
205289
payload that the job was dispatched with. The `payload` has a **maximum size
@@ -438,6 +522,15 @@ The `Task` object supports the following keys:
438522
defined in the resources block. This could be a label of either a
439523
dynamic or a static port.
440524

525+
- `Provider`: Specifies the service registration provider to use for service
526+
registrations. Valid options are either `consul` or `nomad`. All services
527+
within a single task group must utilise the same provider value.
528+
529+
- `Address`: Specifies a custom address to advertise in Consul or Nomad
530+
service registration. If set, `AddressMode` must be in `auto` mode. Useful
531+
with interpolation - for example to advertise the public IP address of an
532+
AWS EC2 instance set this to `${attr.unique.platform.aws.public-ipv4}`.
533+
441534
- `AddressMode`: Specifies what address (host or driver-specific) this
442535
service should advertise. This setting is supported in Docker since
443536
Nomad 0.6 and rkt since Nomad 0.7. Valid options are:

‎website/content/docs/commands/job/run.mdx

+15-4
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,10 @@ that volume.
7474
- `-eval-priority`: Override the priority of the evaluations produced as a result
7575
of this job submission. By default, this is set to the priority of the job.
7676

77+
- `-json`: Parses the job file as JSON. If the outer object has a Job field,
78+
such as from "nomad job inspect" or "nomad run -output", the value of the
79+
field is used as the job. See [JSON Jobs] for details.
80+
7781
- `-hcl1`: If set, HCL1 parser is used for parsing the job spec.
7882

7983
- `-hcl2-strict`: Whether an error should be produced from the HCL2 parser where
@@ -225,11 +229,18 @@ $ nomad job run example.nomad
225229
==> 2021-06-14T09:25:09-07:00: Evaluation "88a91284" finished with status "complete"
226230
```
227231

228-
[`go-getter`]: https://github.com/hashicorp/go-getter
229-
[deployment status]: /docs/commands/deployment#status
230232
[`batch`]: /docs/schedulers#batch
231-
[`system`]: /docs/schedulers#system
232-
[`job plan` command]: /docs/commands/job/plan
233+
[`consul` stanza `allow_unauthenticated`]: /docs/configuration/consul#allow_unauthenticated
234+
[deployment status]: /docs/commands/deployment#status
233235
[eval status]: /docs/commands/eval-status
236+
[`go-getter`]: https://github.com/hashicorp/go-getter
237+
[`job plan` command]: /docs/commands/job/plan
234238
[job specification]: /docs/job-specification
239+
<<<<<<< HEAD
235240
[`allow_unauthenticated`]: /docs/configuration/consul#allow_unauthenticated
241+
=======
242+
[JSON jobs]: /api-docs/json-jobs
243+
[`system`]: /docs/schedulers#system
244+
[`vault` stanza `allow_unauthenticated`]: /docs/configuration/vault#allow_unauthenticated
245+
[`vault_token`]: /docs/job-specification/job#vault_token
246+
>>>>>>> e4d6d5103 (docs: update json jobs docs (#12766))

0 commit comments

Comments
 (0)