code review comments

Lightning-AI · Oct 20, 2022 · 5b7ba4e · 5b7ba4e
1 parent 93aa054
commit 5b7ba4e
Show file tree

Hide file tree

Showing 6 changed files with 67 additions and 73 deletions.
diff --git a/docs/source-app/index.rst b/docs/source-app/index.rst
@@ -216,17 +216,18 @@ Keep Learning
    :caption: How to...
 
    Access the App State <workflows/access_app_state/access_app_state>
-   Add a web user interface (UI) <workflows/add_web_ui/index>
    Add a web link  <workflows/add_web_link>
+   Add a web user interface (UI) <workflows/add_web_ui/index>
    Add encrypted secrets <glossary/secrets>
    Arrange app tabs <workflows/arrange_tabs/index>
+   Cache Work run calls  <workflows/run_work_once>
+   Customize your cloud compute <core_api/lightning_work/compute>
    Develop a Command Line Interface (CLI) <workflows/build_command_line_interface/index>
    Develop a Lightning App <workflows/build_lightning_app/index>
    Develop a Lightning Component <workflows/build_lightning_component/index>
    Develop a REST API <workflows/build_rest_api/index>
-   Cache Work run calls  <workflows/run_work_once>
-   Customize your cloud compute <core_api/lightning_work/compute>
    Extend an existing app <workflows/extend_app>
+   Mount Data from a Cloud Object Store to the Filesystem <workflows/mount_cloud_object_store>
    Publish a Lightning component <workflows/build_lightning_component/publish_a_component>
    Run a server within a Lightning App <workflows/add_server/index>
    Run an App on the cloud <workflows/run_app_on_cloud/index>
@@ -235,7 +236,6 @@ Keep Learning
    Save files <glossary/storage/drive.rst>
    Share an app  <workflows/share_app>
    Share files between components <workflows/share_files_between_components>
-   Mount an AWS S3 Bucket to the Filesystem <workflows/mount_aws_s3_bucket>
 
 ..
    [Docs under construction] Add a Lightning component  <workflows/add_components/index>

diff --git a/docs/source-app/workflows/index.rst b/docs/source-app/workflows/index.rst
@@ -109,9 +109,9 @@ Common Workflows
 
 .. displayitem::
    :header: Mount an AWS S3 Bucket to the Filesystem
-   :description: Learn how Lightning Mounts are used to make the contents of an AWS S3 bucket available on disk when running in the cloud.
+   :description: Learn how Lightning Mounts are used to make the contents of an cloud object store bucket available on disk when running in the cloud.
    :col_css: col-md-4
-   :button_link: mount_aws_s3_bucket.html
+   :button_link: mount_cloud_object_store.html
    :height: 180
 
 

diff --git a/...rce-app/workflows/mount_aws_s3_bucket.rst → ...pp/workflows/mount_cloud_object_store.rst b/...rce-app/workflows/mount_aws_s3_bucket.rst → ...pp/workflows/mount_cloud_object_store.rst
@@ -1,61 +1,24 @@
 :orphan:
 
-########################################
-Mount an AWS S3 Bucket to the Filesystem
-########################################
+######################################################
+Mount Data From a Cloud Object Store to the Filesystem
+######################################################
 
-.. note:: The contents of this page is still in progress!
+**Audience:** Users who want to read files stored in a Cloud Object Bucket in an app.
 
-**Audience:** Users who want to read files stored on an AWS S3 Bucket in an app.
+******************************
+Mounting Public AWS S3 Buckets
+******************************
 
-----
-
-****************************
-Why Do I Need To Use AWS S3?
-****************************
-
-In a Lightning App some components can be executed on their own independent pieces of hardware.
-The Amazon Web Service (AWS) Simple Storage Solution (S3) is a fundamental piece of cloud infrastructure
-which acts as an infinitely scalable object store. It is commonly used by organization who wish to
-keep vast amounts of data files available for use by their applications which run on the cloud.
-
-Utilizing AWS S3 within components of a Lighting App allows the application to natively load
-arbitrary files from S3 into the application; this might be training data, model checkpoints,
-or configuration files (among many other use cases).
-
-----
-
-***********
-Limitations
-***********
-
-Currently the following limitations are enforced when using a ``Mount``:
-
-* Mounted files from an S3 bucket are ``read only``. Any modifications, additions, or deletions
-  to files in the mounted directory will not be reflected in the cloud object store.
-* Mounts can only be configured for a ``LightningWork``. Use in ``LightningFlow`` is not currently supported.
-* Only public S3 buckets can be mounted.
-* We enforce a hard limit of ``1,000,000`` objects which we can mount from a particular bucket prefix.
-* The maximum size of an object in the bucket can be no more than ``5 GiB``.
-* If multiple mounts are configured for a single ``Work``, then they must each specify unique ``root_dir``
-  arguments (a unique mount point).
-
-.. warning::
-   If the bucket prefix contains more than ``1,000,000`` objects or a file greater than ``5 GiB`` in size
-   then the ``LightningWork`` with the ``Mount`` configured on it will fail before it begins running on the cloud.
-
-----
-
-*****************************
+=============================
 Configuring a Mount in a Work
-*****************************
+=============================
 
-In order to configure a mounted s3 bucket in a work, you simply set the ``mounts`` field of a ``CloudCompute``
-object which is passed to any ``LightingWork`` class. The ``source`` field indicates the bucket prefix (i.e. folder)
-to use as the data source, and the ``root_dir`` argument is used to specify where the data should appear on the
-filesystem disk.
+To mount data from a cloud bucket to your app compute, initialize a ``Mount`` object with the source path and
+the absolute directory path where it should be mounted and pass it to the ``CloudCompute`` it should be mounted on.
 
-.. code-block::python
+.. code:: python
+    :emphasize-lines: 9-12
 
     import lightning as L
     from lightning_app import CloudCompute
@@ -79,21 +42,20 @@ filesystem disk.
 You can also pass multiple mounts to a single work by passing a ``List[Mount(...), ...]`` to the
 ``CloudCompute(mounts=...)`` argument.
 
-----
-
-***********************
-Accessing Files on Disk
-***********************
+=======================
+Accessing Mounted Files
+=======================
 
-When a mount is configured in for a ``LightningWork`` running in the cloud, the ``root_dir`` directory
-path is automatically created and populated with the data before your ``LightningWork`` class even begins
+When a mount is configured via ``CloudCompute`` for a ``LightningWork`` running in the cloud, the ``root_dir``
+directory path is automatically created and populated with the data before your ``LightningWork`` class even begins
 executing. **The files stored in the AWS S3 bucket appear "automagically" as normal files on your local disk**,
 allowing you to perform any standard inspection, listing, or reading of them with standard file processing
 logic (just like dealing with files on your local machine!)
 
 If we expand on the example above, we can see how you might go about listing and reading a file!
 
-.. code-block::python
+.. code:: python
+    :emphasize-lines: 9-14
 
     import os
 
@@ -129,12 +91,44 @@ If we expand on the example above, we can see how you might go about listing and
 The ``LightningWork`` component in the code above (``MyWorkClass``) would print out a list of files stored
 in the mounted s3 bucket & then read the contents of a file ``"esRedditJson1"``.
 
-____
-
-*****************************
+=============================
 Mounts & Locally Running Apps
-*****************************
+=============================
 
 When running a Lighting App on your local machine, any ``CloudCompute`` configuration (including a ``Mount``)
 is ignored at runtime. If you need access to these files on your local disk, you should download a copy of them
 to your machine.
+
+===========
+Limitations
+===========
+
+Currently the following limitations are enforced when using a ``Mount``:
+
+* Mounted files from an S3 bucket are ``read only``. Any modifications, additions, or deletions
+  to files in the mounted directory will not be reflected in the cloud object store.
+* Mounts can only be configured for a ``LightningWork``. Use in ``LightningFlow`` is not currently supported.
+* We enforce a hard limit of ``1,000,000`` objects which we can mount from a particular bucket prefix.
+* The maximum size of an object in the bucket can be no more than ``5 GiB``.
+* If multiple mounts are configured for a single ``Work``, then they must each specify unique ``root_dir``
+  arguments (a unique mount point).
+
+.. note::
+   If the bucket prefix contains more than ``1,000,000`` objects or a file greater than ``5 GiB`` in size
+   then the ``LightningWork`` will fail to start before it even begins running on the cloud.
+
+----
+
+**********************************************
+Mounting Private AWS S3 Buckets - Coming Soon!
+**********************************************
+
+We'll Let you know when this feature is ready!
+
+----
+
+************************************************
+Mounting Google Cloud GCS Buckets - Coming Soon!
+************************************************
+
+We'll Let you know when this feature is ready!
diff --git a/docs/source-lit/index.rst b/docs/source-lit/index.rst
@@ -62,15 +62,16 @@ Welcome to ⚡ Lightning Apps
    :caption: How to...
 
    Access the App State <workflows/access_app_state/access_app_state>
-   Add a web user interface (UI) <workflows/add_web_ui/index>
    Add a web link  <workflows/add_web_link>
+   Add a web user interface (UI) <workflows/add_web_ui/index>
    Add encrypted secrets <glossary/secrets>
    Arrange app tabs <workflows/arrange_tabs/index>
-   Develop a Lightning App <workflows/build_lightning_app/index>
-   Develop a Lightning Component <workflows/build_lightning_component/index>
    Cache Work run calls  <workflows/run_work_once>
    Customize your cloud compute <core_api/lightning_work/compute>
+   Develop a Lightning App <workflows/build_lightning_app/index>
+   Develop a Lightning Component <workflows/build_lightning_component/index>
    Extend an existing app <workflows/extend_app>
+   Mount Data from a Cloud Object Store to the Filesystem <workflows/mount_cloud_object_store>
    Publish a Lightning component <workflows/build_lightning_component/publish_a_component>
    Run a server within a Lightning App <workflows/add_server/index>
    Run an App on the cloud <workflows/run_app_on_cloud/index>
@@ -79,7 +80,6 @@ Welcome to ⚡ Lightning Apps
    Save files <glossary/storage/drive.rst>
    Share an app  <workflows/share_app>
    Share files between components <workflows/share_files_between_components>
-   Mount an AWS S3 Bucket to the Filesystem <workflows/mount_aws_s3_bucket>
 
 ..
    [Docs under construction] Add a Lightning component  <workflows/add_components/index>

diff --git a/docs/source-lit/workflows/mount_aws_s3_bucket.rst b/docs/source-lit/workflows/mount_aws_s3_bucket.rst
diff --git a/docs/source-lit/workflows/mount_cloud_object_store.rst b/docs/source-lit/workflows/mount_cloud_object_store.rst
@@ -0,0 +1 @@
+../../source-app/workflows/mount_cloud_object_store.rst
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		../../source-app/workflows/mount_cloud_object_store.rst