added docs for Mount object

docs/source-app/api_reference/storage.rst

@@ -20,6 +20,7 @@ ______________
     ~path.Path
     ~drive.Drive
     ~payload.Payload
+    ~mount.Mount
 
 ----
 
@@ -56,6 +57,14 @@ Learn more about Storage
    :height: 180
    :tag: Intermediate
 
+.. displayitem::
+   :header: The Mount Object.
+   :description: Mount an AWS S3 Bucket When Running on the Cloud.
+   :col_css: col-md-4
+   :button_link:  ../workflows/mount_aws_s3_bucket.html
+   :height: 180
+   :tag: Intermediate
+
 .. raw:: html
 
         </div>

docs/source-app/api_references.rst

@@ -71,6 +71,7 @@ _______
     ~path.Path
     ~drive.Drive
     ~payload.Payload
+    ~mount.Mount
 
 Learn more about :ref:`Storage <storage>`.
 
@@ -89,3 +90,19 @@ _______
     ~cloud.CloudRuntime
     ~singleprocess.SingleProcessRuntime
     ~multiprocess.MultiProcessRuntime
+
+
+----
+
+lightning_app.utilities.packaging
+_________________________________
+
+.. currentmodule:: lightning_app.utilities.packaging
+
+.. autosummary::
+    :toctree: generated/
+    :nosignatures:
+    :template: classtemplate_no_index.rst
+
+    ~cloud_compute.CloudCompute
+    ~build_config.BuildConfig

docs/source-app/index.rst

@@ -235,6 +235,7 @@ Keep Learning
    Save files <glossary/storage/drive.rst>
    Share an app  <workflows/share_app>
    Share files between components <workflows/share_files_between_components>
+   Mount an AWS S3 Bucket to the Filesystem <workflows/mount_aws_s3_bucket>
 
 ..
    [Docs under construction] Add a Lightning component  <workflows/add_components/index>
@@ -260,6 +261,7 @@ Keep Learning
    api_reference/frontend
    api_reference/runners
    api_reference/storage
+   api_reference/compute_config
 
 .. toctree::
    :maxdepth: 1

docs/source-app/workflows/mount_aws_s3_bucket.rst

@@ -13,8 +13,9 @@ Mount an AWS S3 Bucket to the Filesystem
 ****************************
 Why Do I Need To Use AWS S3?
 ****************************
+
 In a Lightning App some components can be executed on their own independent pieces of hardware.
-The Amazon Web Service (AWS) Simple Storage Solution (S3) is a fundumental piece of cloud infrastructure
+The Amazon Web Service (AWS) Simple Storage Solution (S3) is a fundamental piece of cloud infrastructure
 which acts as an infinitely scalable object store. It is commonly used by organization who wish to
 keep vast amounts of data files available for use by their applications which run on the cloud.
 
@@ -28,21 +29,112 @@ or configuration files (among many other use cases).
 Limitations
 ***********
 
+Currently the following limitations are enforced when using a ``Mount``:
+
+* Mounted files from an S3 bucket are ``read only``. Any modifications, additions, or deletions
+  to files in the mounted directory will not be reflected in the cloud object store.
+* Mounts can only be configured for a ``LightningWork``. Use in ``LightningFlow`` is not currently supported.
+* Only public S3 buckets can be mounted.
+* We enforce a hard limit of ``1,000,000`` objects which we can mount from a particular bucket prefix.
+* The maximum size of an object in the bucket can be no more than ``5 GiB``.
+* If multiple mounts are configured for a single ``Work``, then they must each specify unique ``root_dir``
+  arguments (a unique mount point).
+
+.. warning::
+   If the bucket prefix contains more than ``1,000,000`` objects or a file greater than ``5 GiB`` in size
+   then the ``LightningWork`` with the ``Mount`` configured on it will fail before it begins running on the cloud.
 
 ----
 
-*************************
-Passing a Mount to a Work
-*************************
+*****************************
+Configuring a Mount in a Work
+*****************************
+
+In order to configure a mounted s3 bucket in a work, you simply set the ``mounts`` field of a ``CloudCompute``
+object which is passed to any ``LightingWork`` class. The ``source`` field indicates the bucket prefix (i.e. folder)
+to use as the data source, and the ``root_dir`` argument is used to specify where the data should appear on the
+filesystem disk.
+
+.. code-block::python
+
+    import lightning as L
+    from lightning_app import CloudCompute
+    from lightning_app.storage import Mount
+
+    class Flow(L.LightningFlow):
+       def __init__(self):
+           super().__init__()
+           self.my_work = MyWorkClass(
+               cloud_compute=CloudCompute(
+                   mounts=Mount(
+                       source="s3://ryft-public-sample-data/esRedditJson/",
+                       root_dir="/content/esRedditJson/",
+                   ),
+               )
+           )
+
+       def run(self):
+           self.my_work.run()
+
+You can also pass multiple mounts to a single work by passing a ``List[Mount(...), ...]`` to the
+``CloudCompute(mounts=...)`` argument.
 
 ----
 
 ***********************
 Accessing Files on Disk
 ***********************
 
-----
+When a mount is configured in for a ``LightningWork`` running in the cloud, the ``root_dir`` directory
+path is automatically created and populated with the data before your ``LightningWork`` class even begins
+executing. **The files stored in the AWS S3 bucket appear "automagically" as normal files on your local disk**,
+allowing you to perform any standard inspection, listing, or reading of them with standard file processing
+logic (just like dealing with files on your local machine!)
+
+If we expand on the example above, we can see how you might go about listing and reading a file!
+
+.. code-block::python
+
+    import os
+
+    import lightning as L
+    from lightning_app import CloudCompute
+    from lightning_app.storage import Mount
+
+    class MyWorkClass(L.LightningWork):
+       def run(self):
+           files = os.listdir("/content/esRedditJson/")
+           for file in files:
+               print(file)
+
+           with open("/content/esRedditJson/esRedditJson1", "r") as f:
+               some_data = f.read()
+               # do something with "some_data"...
+
+    class Flow(L.LightningFlow):
+       def __init__(self):
+           super().__init__()
+           self.my_work = MyWorkClass(
+               cloud_compute=CloudCompute(
+                   mounts=Mount(
+                       source="s3://ryft-public-sample-data/esRedditJson/",
+                       root_dir="/content/esRedditJson/",
+                   ),
+               )
+           )
+
+       def run(self):
+           self.my_work.run()
+
+The ``LightningWork`` component in the code above (``MyWorkClass``) would print out a list of files stored
+in the mounted s3 bucket & then read the contents of a file ``"esRedditJson1"``.
+
+____
+
+*****************************
+Mounts & Locally Running Apps
+*****************************
 
-*********************
-Using Multiple Mounts
-*********************
+When running a Lighting App on your local machine, any ``CloudCompute`` configuration (including a ``Mount``)
+is ignored at runtime. If you need access to these files on your local disk, you should download a copy of them
+to your machine.

docs/source-lit/api_reference/api/lightning_app.core.LightningApp.rst

@@ -0,0 +1,10 @@
+.. role:: hidden
+    :class: hidden-section
+.. currentmodule:: lightning_app.core
+
+
+LightningApp
+============
+
+.. autoclass:: LightningApp
+    :members:

docs/source-lit/api_reference/api/lightning_app.core.LightningFlow.rst

@@ -0,0 +1,10 @@
+.. role:: hidden
+    :class: hidden-section
+.. currentmodule:: lightning_app.core
+
+
+LightningFlow
+=============
+
+.. autoclass:: LightningFlow
+    :members:

docs/source-lit/api_reference/api/lightning_app.core.LightningWork.rst

@@ -0,0 +1,10 @@
+.. role:: hidden
+    :class: hidden-section
+.. currentmodule:: lightning_app.core
+
+
+LightningWork
+=============
+
+.. autoclass:: LightningWork
+    :members:

docs/source-lit/api_reference/compute_config.rst

@@ -0,0 +1 @@
+../../source-app/api_reference/compute_config.rst

docs/source-lit/index.rst

@@ -79,6 +79,7 @@ Welcome to ⚡ Lightning Apps
    Save files <glossary/storage/drive.rst>
    Share an app  <workflows/share_app>
    Share files between components <workflows/share_files_between_components>
+   Mount an AWS S3 Bucket to the Filesystem <workflows/mount_aws_s3_bucket>
 
 ..
    [Docs under construction] Add a Lightning component  <workflows/add_components/index>
@@ -104,6 +105,7 @@ Welcome to ⚡ Lightning Apps
    api_reference/frontend
    api_reference/runners
    api_reference/storage
+   api_reference/compute_config
 
 .. toctree::
    :maxdepth: 1

docs/source-lit/workflows/mount_aws_s3_bucket.rst

@@ -0,0 +1 @@
+../../source-app/workflows/mount_aws_s3_bucket.rst