Merge pull request #1466 from hashicorp/d-logs

Document logs command and API

Author: dadgar

command/logs.go

@@ -28,6 +28,9 @@ General Options:
 
 Logs Specific Options:
 
+  -stderr:
+    Display stderr logs.
+
   -verbose
     Show full information.
 
@@ -39,15 +42,15 @@ Logs Specific Options:
     rather to wait for additional output.
 
   -tail
-    Show the files contents with offsets relative to the end of the file. If no
+    Show the logs contents with offsets relative to the end of the logs. If no
     offset is given, -n is defaulted to 10.
 
   -n
     Sets the tail location in best-efforted number of lines relative to the end
-    of the file.
+    of the logs.
 
   -c
-    Sets the tail location in number of bytes relative to the end of the file.
+    Sets the tail location in number of bytes relative to the end of the logs.
 	`
 	return strings.TrimSpace(helpText)
 }

website/source/docs/commands/fs.html.md.erb

@@ -68,22 +68,22 @@ drwxrwxr-x  4096  28 Jan 16 05:39 UTC  redis/
 -rw-rw-r--  0     28 Jan 16 05:39 UTC  redis_exit_status
 
 
-$ nomad fs redis/local
+$ nomad fs eb17e557 redis/local
 Mode        Size  Modified Time        Name
 -rw-rw-rw-  0     28 Jan 16 05:39 UTC  redis.stderr
 -rw-rw-rw-  17    28 Jan 16 05:39 UTC  redis.stdout
 
 
-$ nomad fs -stat redis/local/redis.stdout
+$ nomad fs -stat eb17e557 redis/local/redis.stdout
 Mode        Size  Modified Time        Name
 -rw-rw-rw-  17    28 Jan 16 05:39 UTC  redis.stdout
 
 
-$ nomad fs redis/local/redis.stdout
+$ nomad fs eb17e557 redis/local/redis.stdout
 foobar
 baz
 
-$ nomad fs -tail -f -n 3 redis/local/redis.stdout
+$ nomad fs -tail -f -n 3 eb17e557 redis/local/redis.stdout
 foobar
 baz
 bam
@@ -92,15 +92,15 @@ bam
 
 ## Using Job ID instead of Allocation ID
 
-Passing `-job` into one of the `fs` commands will allow the `fs` command to
-randomly select an allocation ID from the specified job.
+Setting the `-job` flag causes a random allocation of the specified job to be
+selected. Nomad will prefer to select a running allocation ID for the job, but
+if no running allocations for the job are found, Nomad will use a dead
+allocation.
 
 ```
 nomad fs -job <job-id> <path>
 ```
 
-Nomad will prefer to select a running allocation ID for the job, but if no
-running allocations for the job are found, Nomad will use a dead allocation.
 
 This can be useful for debugging a job that has multiple allocations, and it's
 not really required to use a specific allocation ID.

website/source/docs/commands/logs.html.md.erb

@@ -0,0 +1,88 @@
+---
+layout: "docs"
+page_title: "Commands: logs"
+sidebar_current: "docs-commands-logs"
+description: >
+  Stream the logs of a task.
+---
+
+# Command: logs
+
+The `logs` command displays the log of a given task.
+
+## Usage
+
+```
+nomad logs [options] <alloc-id> <task>
+```
+
+This commands streams the logs of the given task in the allocation. If the
+allocation is only running a single task, the task name can be omitted.
+Optionally, the `-job` option may be used in which case a random allocation from
+the given job will be chosen.
+#
+## General Options
+
+<%= general_options_usage %>
+
+## Logs Options
+
+* `-stderr`: Display stderr logs.
+
+* `-verbose`: Display verbose output.
+
+* `-job`: Use a random allocation from the specified job, prefering a running
+allocation.
+
+* `-f`: Causes the output to not stop when the end of the logs are reached, but
+rather to wait for additional output. 
+
+* `-tail`: Show the logs contents with offsets relative to the end of the logs.
+If no offset is given, -n is defaulted to 10.
+
+* `-n`: Sets the tail location in best-efforted number of lines relative to the
+end of the logs.
+
+* `-c`: Sets the tail location in number of bytes relative to the end of the logs.
+
+## Examples
+
+```
+$ nomad logs eb17e557 redis
+foobar
+baz
+bam
+
+$ nomad logs -stderr eb17e557 redis
+[ERR]: foo
+[ERR]: bar
+
+$ nomad logs -job example
+[ERR]: foo
+[ERR]: bar
+
+$ nomad logs -tail -n 2 eb17e557 redis
+foobar
+baz
+
+$ nomad logs -tail -f -n 3 eb17e557 redis
+foobar
+baz
+bam
+<blocking>
+```
+
+## Using Job ID instead of Allocation ID
+
+Setting the `-job` flag causes a random allocation of the specified job to be
+selected. Nomad will prefer to select a running allocation ID for the job, but
+if no running allocations for the job are found, Nomad will use a dead
+allocation.
+
+```
+nomad logs -job <job-id> <task>
+```
+
+
+This can be useful for debugging a job that has multiple allocations, and it's
+not really required to use a specific allocation ID.

website/source/docs/http/client-fs.html.md

@@ -133,6 +133,95 @@ allocation was placed.
   </dd>
 </dl>
 
+<a id="logs"></a>
+
+<dl>
+  <dt>Description</dt>
+  <dd>
+     Stream a tasks stdout/stderr logs.
+  </dd>
+
+  <dt>Method</dt>
+  <dd>GET</dd>
+
+  <dt>URL</dt>
+  <dd>`/v1/client/fs/logs/<Allocation-ID>`</dd>
+
+  <dt>Parameters</dt>
+  <dd>
+    <ul>
+      <li>
+        <span class="param">task</span>
+        <span class="param-flags">required</span>
+        The name of the task inside the allocation to stream logs from.
+      </li>
+      <li>
+        <span class="param">follow</span>
+        <span class="param-flags">required</span>
+         A boolean of whether to follow logs.
+      </li>
+      <li>
+        <span class="param">type</span>
+         Either, "stdout" or "stderr", defaults to "stdout" if omitted.
+      </li>
+      <li>
+        <span class="param">offset</span>
+         The offset to start streaming from. Defaults to 0.
+      </li>
+      <li>
+        <span class="param">origin</span>
+        Origin can be either "start" or "end" and applies the offset relative to
+        either the start or end of the logs respectively. Defaults to "start".
+      </li>
+    </ul>
+  </dd>
+
+  <dt>Returns</dt>
+  <dd>
+
+    ```
+...
+    {
+        "File":"alloc/logs/redis.stdout.0",
+        "Offset":3604480
+        "Data": "NTMxOTMyCjUzMTkzMwo1MzE5MzQKNTMx..."
+    }
+    {
+        "File":"alloc/logs/redis.stdout.0",
+        "FileEvent": "file deleted"
+    }
+    ```
+
+  </dd>
+
+
+  <dt>Field Reference</dt>
+  <dd>
+    The return value is a stream of frames. These frames contain the following
+    fields:
+
+    <ul>
+      <li>
+        <span class="param">Data</span>
+        A base64 encoding of the bytes being streamed.
+      </li>
+      <li>
+        <span class="param">FileEvent</span>
+        An event that could cause a change in the streams position. The possible
+        values are "file deleted" and "file truncated".
+      </li>
+      <li>
+        <span class="param">Offset</span>
+        Offset is the offset into the stream.
+      </li>
+      <li>
+        <span class="param">File</span>
+        The name of the file being streamed.
+      </li>
+    </ul>
+  </dd>
+</dl>
+
 <dl>
   <dt>Description</dt>
   <dd>

website/source/docs/jobops/logs.html.md

@@ -10,8 +10,10 @@ description: |-
 
 Accessing applications logs is critical when debugging issues, performance
 problems or even for verifying the application is starting correctly. To make
-this as simple as possible, Nomad provides both a CLI tool and an API for
-accessing application logs and data files.
+this as simple as possible, Nomad provides [log
+rotation](/docs/jobspec/index.html#log_rotation) in the jobspec, provides a [CLI
+command](/docs/commands/logs.html) and an [API](/docs/http/client-fs.html#logs)
+for accessing application logs and data files.
 
 To see this in action we can just run the example job which created using `nomad
 init`:
@@ -32,18 +34,15 @@ $ nomad run example.nomad
 ==> Evaluation "7a3b78c0" finished with status "complete"
 ```
 
-We can grab the allocation ID from above and use the [`nomad fs`
-command](/docs/commands/fs.html) to access the applications logs. Logs are
-stored under the following directory structure:
-`alloc/logs/<task-name>.<stdout/stderr>.<index>`. Nomad has built in log
-rotation, documented in the [Jobspec](/docs/jobspec/index.html#log_rotation).
-The index is a monotonically increasing number starting at zero and incremented
-each time the log is rotated.
+We can grab the allocation ID from above and use the [`nomad logs`
+command](/docs/commands/logs.html) to access the applications logs. The `logs`
+command supports both displaying the logs as well as following logs, blocking
+for more output. 
 
 Thus to access the `stdout` we can issue the below command:
 
 ```
-$ nomad fs c3c58508 alloc/logs/redis.stdout.0
+$ nomad logs c3c58508 redis
                  _._
             _.-``__ ''-._
        _.-``    `.  `_.  ''-._           Redis 3.2.1 (00000000/0) 64 bit
@@ -69,7 +68,11 @@ $ nomad fs c3c58508 alloc/logs/redis.stdout.0
  1:M 28 Jun 19:49:30.505 * The server is now ready to accept connections on port 6379
 ```
 
-Replacing `stdout` for `stderr` would display the respective `stderr` output.
+To display the `stderr` for the task we would run the following: 
+
+```
+$ nomad logs -stderr c3c58508 redis
+```
 
 While this works well for quickly accessing logs, we recommend running a
 log-shipper for long term storage of logs. In many cases this will not be needed

website/source/intro/getting-started/install.html.md

@@ -55,6 +55,7 @@ Available commands are:
     fs                    Inspect the contents of an allocation directory
     init                  Create an example job file
     inspect               Inspect a submitted job
+    logs                  Streams the logs of a task.
     node-drain            Toggle drain mode on a given node
     node-status           Display status information about nodes
     plan                  Dry-run a job update to determine its effects

website/source/intro/getting-started/jobs.html.md

@@ -105,16 +105,10 @@ We can see that Nomad reports the state of the allocation as well as its
 current resource usage. By supplying the `-stats` flag, more detailed resource
 usage statistics will be reported.
 
-To inspect the file system of a running allocation, we can use the [`fs`
-command](/docs/commands/fs.html):
+To see the logs of a task, we can use the [logs command](/docs/commands/logs.html):
 
 ```
-$ nomad fs 8ba85cef alloc/logs
-Mode        Size    Modified Time          Name
--rw-rw-r--  0 B     15/03/16 15:40:56 PDT  redis.stderr.0
--rw-rw-r--  2.3 kB  15/03/16 15:40:57 PDT  redis.stdout.0
-
-$ nomad fs 8ba85cef alloc/logs/redis.stdout.0
+$ nomad logs 8ba85cef redis
                  _._
             _.-``__ ''-._
        _.-``    `.  `_.  ''-._           Redis 3.2.1 (00000000/0) 64 bit

website/source/intro/getting-started/next-steps.html.md

@@ -20,3 +20,7 @@ We recommend reading the following as next steps.
 
   * [Creating a Cluster](/docs/cluster/bootstrapping.html) - Additional details on
     creating a production worthy Nomad Cluster.
+
+  * [Operating a Job](/docs/jobops/index.html) - Additional details on how to
+    run a job in production.
+

website/source/layouts/docs.erb

@@ -157,6 +157,9 @@
 						<li<%= sidebar_current("docs-commands-inspect") %>>
 							<a href="/docs/commands/inspect.html">inspect</a>
 						</li>
+						<li<%= sidebar_current("docs-commands-logs") %>>
+							<a href="/docs/commands/logs.html">logs</a>
+						</li>
 						<li<%= sidebar_current("docs-commands-node-drain") %>>
 							<a href="/docs/commands/node-drain.html">node-drain</a>
 						</li>