Feature/enhancements cherrypick 2 (#342)

* Update DATABASE.adoc to load about us fixture

* Update docs

Co-authored-by: ayeshamk <[email protected]>

DATABASE.adoc

@@ -124,6 +124,14 @@ cd /path/to/FlatGov/server_py
 pip install -r requirements.txt
 ```
 
+## Load fixtures
+
+The About page is created as a fixture. To load it, run
+```
+cd server_py/flatgov
+python manage.py loaddata about_us.json
+```
+
 ## Create billsMeta.json file via Django command.
 
 * Add data to database

server_py/flatgov/about_us.json

@@ -0,0 +1 @@
+[{"model": "home.aboutpage", "pk": 1, "fields": {"title": "About page content", "content": "<p>Flatgov is a project of <a href=\"https://demandprogresseducationfund.org/\">Demand Progress Education Fund</a> in partnership with GovTrack.us. Its purpose is to demonstrate how we can use existing information to provide additional context and new insights into the legislative process that are useful for congressional experts.</p>\r\n\r\n<p>Flatgov was built through a human-centered design process intended to surface the needs of different kinds of expert users that interact with legislative information.&nbsp;</p>\r\n\r\n<p>The key insight behind Flatgov is that legislating is an iterative process. Legislation embodies ideas that arise over multiple Congresses and in multiple contexts.&nbsp;</p>\r\n\r\n<p>For example, similar or identical legislation is reintroduced every Congress. How do we identify those identical or similar bills when they reappear?&nbsp;</p>\r\n\r\n<p>In addition, the same legislative idea can be embodied in multiple bills in the same Congress. Sometimes a smaller bill is rolled into larger legislation. How do we know that the larger bill carries legislative ideas that originated in other legislation?</p>\r\n\r\n<p>When we connect identical or similar bills across the same Congress or across multiple Congresses, we gain a greater understanding of the strengths and weaknesses of the legislation, its likely supporters and opponents, and surface the discussion around those legislative ideas.</p>\r\n\r\n<p>For example, if you can connect related legislation, you can gain important insights:</p>\r\n\r\n<ul>\r\n\t<li>Who has supported the legislation in the past and might be willing to support it again?&nbsp;</li>\r\n\t<li>Are the legislation&rsquo;s supporters in a position to help it move through the committee process or in the other chamber?</li>\r\n\t<li>Which committees have considered the legislation and what information was surfacing through their review?&nbsp;</li>\r\n\t<li>What have the legislative support agencies said about the legislation &mdash; whether in Congressional Research Service Reports, Congressional Budget Office reports, or elsewhere?</li>\r\n\t<li>Has the administration weighed in on the legislation and what have they said?</li>\r\n\t<li>Who is responsible for an idea &mdash; and thus deserves credit for it &mdash; that became law through an alternative legislative vehicle?</li>\r\n\t<li>Where is the legislation in the political process and what is likely to happen next?</li>\r\n</ul>\r\n\r\n<p>The work to create Flatgov proceeded along several parallel tracks.</p>\r\n\r\n<ul>\r\n\t<li><strong>We built an engine that allows us to identify identical, similar, and related legislation</strong> based upon the text similarities and also are bootstrapping those relationships identified by the Congressional Research Service.</li>\r\n\t<li><strong>We scraped information from new sources and extracted insights.</strong> For example, we scraped all of the Statements of Administration Policy and identified the bills to which they refer. We also parsed all of the CRS reports to identify when they refer to specific legislation and connected the bills to the reports.</li>\r\n\t<li><strong>We designed our interface to maximize insights provided to its users.</strong> For example, we created a heuristic to sort the list of co-sponsors to identify those most relevant to moving legislation. We also made it easier for staffers to identify members who had supported legislation in the past so they could be prioritized for outreach.</li>\r\n\t<li><span style=\"font-size:16px\"><strong>We made important information downloadable</strong></span> so that users could reuse and enhance the data for their own purposes.</li>\r\n</ul>\r\n\r\n<p>We are publishing the code for Flatgov online for free, without a license, for anyone to use. Our hope is to spur others to repurpose the code for their own uses. We also seek to encourage the Library of Congress to enhance their premier website, Congress.gov, to maximize the value that people can derive from information on their website by putting legislation in the appropriate temporal context and supporting users in their efforts to connect the dots among legislative actions.</p>\r\n\r\n<p>Flatgov was supposed through a one-time grant. There is more work to do, but we hope our work thus far points the way forward.</p>\r\n\r\n<p>&nbsp;</p>", "created_at": "2021-04-30T18:02:02.777Z", "updated_at": "2021-04-30T18:50:24.597Z"}}]

server_py/flatgov/flatgov/settings.py

@@ -60,13 +60,17 @@
 
     'bills',
     'uscongress',
+    'home',
 
     'rest_framework',
     'django_tables2',
     'admin_auto_filters',
     'crs',  # by Dmitry
     'statementAdminPolicy',
     'committeeReport',
+    'ckeditor',
+    "ckeditor_uploader",
+
 ]
 
 MIDDLEWARE = [
@@ -219,4 +223,85 @@
 
 BILL_SUMMARY_DEFAULT_TEXT = 'Bill Summary Default Text'
 
-PROPUBLICA_CONGRESS_API_KEY = os.getenv('PROPUBLICA_CONGRESS_API_KEY', "")
+PROPUBLICA_CONGRESS_API_KEY = os.getenv('PROPUBLICA_CONGRESS_API_KEY', "")
+
+
+
+CKEDITOR_UPLOAD_PATH = "media/"
+CKEDITOR_IMAGE_BACKEND = "pillow"
+CKEDITOR_THUMBNAIL_SIZE = (300, 300)
+CKEDITOR_IMAGE_QUALITY = 40
+CKEDITOR_BROWSE_SHOW_DIRS = True
+CKEDITOR_ALLOW_NONIMAGE_FILES = True
+
+
+# CKEDITOR
+CKEDITOR_CONFIGS = {
+    'default': {
+        'skin': 'moono-lisa',
+        # 'skin': 'office2013',
+        'toolbar_Basic': [
+            ['Source', '-', 'Bold', 'Italic']
+        ],
+        'toolbar_Full': [
+            {'name': 'document', 'items': ['Source', '-', 'Save', 'NewPage', 'Preview', 'Print', '-', 'Templates']},
+            {'name': 'clipboard', 'items': ['Cut', 'Copy', 'Paste', 'PasteText', 'PasteFromWord', '-', 'Undo', 'Redo']},
+            {'name': 'editing', 'items': ['Link', 'Anchor', 'Find', 'Replace', '-', 'SelectAll']},
+            # {'name': 'forms',
+            #  'items': ['Form', 'Checkbox', 'Radio', 'TextField', 'Textarea', 'Select', 'Button', 'ImageButton',
+            #            'HiddenField']},
+            '/',
+            {'name': 'basicstyles',
+             'items': ['Bold', 'Italic', 'Underline', 'Strike', 'Subscript', 'Superscript', '-', 'RemoveFormat']},
+            {'name': 'paragraph',
+             'items': ['NumberedList', 'BulletedList', '-', 'Outdent', 'Indent', '-', 'Blockquote', 'CreateDiv', '-',
+                       'JustifyLeft', 'JustifyCenter', 'JustifyRight', 'JustifyBlock', '-', 'BidiLtr', 'BidiRtl',
+                       'Language']},
+            {'name': 'links', 'items': ['Link', 'Unlink', 'Anchor']},
+            {'name': 'insert',
+             'items': ['Image','Table', 'HorizontalRule','SpecialChar', 'PageBreak', 'Iframe']},
+            '/',
+            {'name': 'styles', 'items': ['Styles', 'Format', 'Font', 'FontSize']},
+            {'name': 'colors', 'items': ['TextColor', 'BGColor']},
+            {'name': 'tools', 'items': ['Maximize', 'ShowBlocks']},
+            {'name': 'about', 'items': ['About']},
+            '/',  # put this to force next toolbar on new line
+            {'name': 'yourcustomtools', 'items': [
+                # put the name of your editor.ui.addButton here
+                'Preview',
+                'Maximize',
+
+            ]},
+        ],
+        'toolbar': 'Full',  # put selected toolbar config here
+        # 'toolbarGroups': [{ 'name': 'document', 'groups': [ 'mode', 'document', 'doctools' ] }],
+        "height": 291,
+        "width": 835,
+        'filebrowserWindowHeight': 725,
+        'filebrowserWindowWidth': 940,
+        # 'toolbarCanCollapse': True,
+        # 'mathJaxLib': '//cdn.mathjax.org/mathjax/2.2-latest/MathJax.js?config=TeX-AMS_HTML',
+        'tabSpaces': 4,
+        'extraPlugins': ','.join([
+            "image2",
+            "codesnippet",
+            # 'uploadimage',  # the upload image feature
+            # your extra plugins here
+            'div',
+            'autolink',
+            'autoembed',
+            'embedsemantic',
+            'autogrow',
+            # 'devtools',
+            'widget',
+            'lineutils',
+            'clipboard',
+            'dialog',
+            'dialogui',
+            'elementspath',
+
+        ]),
+        "removePlugins": ",".join(["image"]),
+        "codeSnippet_theme": "xcode",
+    }
+}

server_py/flatgov/flatgov/urls.py

@@ -21,7 +21,7 @@
 urlpatterns = [
     re_path(r'^$', RedirectView.as_view(url='home/', permanent=False), name='index'),
     path('bills/', include('bills.urls')),
-    path('home/', include('home.urls')),
+    path('', include('home.urls')),
     path('uscongress/debug/', include('uscongress.urls')),
     path('admin/', admin.site.urls),
     path('crs/', include('crs.urls')),

server_py/flatgov/home/admin.py

@@ -1,3 +1,18 @@
 from django.contrib import admin
 
-# Register your models here.
+from .models import (
+    AboutPage
+)
+
+
+class AboutPageAdmin(admin.ModelAdmin):
+    class Meta:
+        model = AboutPage
+
+    def has_add_permission(self, request):
+        if self.model.objects.count() >= 1:
+            return False
+        return super().has_add_permission(request)
+
+
+admin.site.register(AboutPage, AboutPageAdmin)

server_py/flatgov/home/migrations/0001_initial.py

@@ -0,0 +1,25 @@
+# Generated by Django 3.1 on 2021-04-30 17:43
+
+import ckeditor.fields
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    initial = True
+
+    dependencies = [
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='AboutPage',
+            fields=[
+                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('title', models.CharField(blank=True, default='About page content', max_length=127, null=True)),
+                ('content', ckeditor.fields.RichTextField()),
+                ('created_at', models.DateTimeField(auto_now_add=True)),
+                ('updated_at', models.DateTimeField(auto_now=True)),
+            ],
+        ),
+    ]

server_py/flatgov/home/models.py

@@ -1,3 +1,12 @@
 from django.db import models
+from ckeditor.fields import RichTextField
 
-# Create your models here.
+class AboutPage(models.Model):
+    title = models.CharField(max_length=127, null=True, blank=True, default='About page content')
+    content = RichTextField()
+
+    created_at = models.DateTimeField(auto_now_add=True)
+    updated_at = models.DateTimeField(auto_now=True)
+
+    def __str__(self):
+        return f'{self.title} updated on {self.updated_at}'

server_py/flatgov/home/urls.py

@@ -4,19 +4,22 @@
 
 from . import views
 
+app_name = 'home'
+
 urlpatterns = [
     # path('', views.index, name='index'),
     path(
         "robots.txt",
         TemplateView.as_view(template_name="robots.txt", content_type="text/plain"),
     ),
-    path('', views.home_view, name='home'),
+    path('home/', views.home_view, name='home'),
+    path('about/', views.AboutView.as_view(), name='about'),
 ]
 
 api_urlpatterns = [
-    path('bill-list/', never_cache(views.BillListAPIView.as_view()), name='bill-list'),
-    path('bill-titles/<congressnumber>', never_cache(views.BillListTitleAPIView.as_view()), name='bill-titles'),
-    path('bill-title/<bill>/', never_cache(views.GetBillTitleAPIView.as_view()), name='bill-title'),
+    path('home/bill-list/', never_cache(views.BillListAPIView.as_view()), name='bill-list'),
+    path('home/bill-titles/<congressnumber>', never_cache(views.BillListTitleAPIView.as_view()), name='bill-titles'),
+    path('home/bill-title/<bill>/', never_cache(views.GetBillTitleAPIView.as_view()), name='bill-title'),
 ]
 
 urlpatterns += api_urlpatterns

server_py/flatgov/home/views.py

@@ -5,11 +5,13 @@
 from django.http import HttpResponse, JsonResponse
 from django.db.models.functions import Concat
 from django.views import View
+from django.views.generic import TemplateView
 
 from common.constants import START_CONGRESS, CURRENT_CONGRESS
 
 from home.forms import QueryForm
 from bills.models import Bill
+from .models import AboutPage
 
 def index(request):
     return HttpResponse("Hello, world. You're at the home index.")
@@ -31,6 +33,14 @@ def home_view(request):
     context = {'form': form, 'congressrange': list(reversed(list(range(START_CONGRESS, CURRENT_CONGRESS +1))))}
     return render(request, 'home/home.html', context)
 
+class AboutView(TemplateView):
+    template_name = 'home/about.html'
+
+    def get_context_data(self, **kwargs):
+        context = super(AboutView, self).get_context_data(**kwargs)
+        context['about'] = AboutPage.objects.first()
+        return context
+
 
 class BillListAPIView(View):
 

server_py/flatgov/static/css/style.css

@@ -289,6 +289,7 @@ span.twitter-typeahead .tt-suggestion:focus {
   /* Tooltip text */
   /* See https://www.w3schools.com/css/css_tooltip.asp */
   .toolt .tooltiptext {
+    font-size: 14px;
     visibility: hidden;
     width: 500px;
     background-color: rgb(165, 165, 165);

server_py/flatgov/static/docs/BILL_MODEL.html

@@ -641,7 +641,7 @@ <h3 id="_related_fields">Related Fields</h3>
 </div>
 <div id="footer">
 <div id="footer-text">
-Last updated 2021-04-30 15:04:03 -0700
+Last updated 2021-05-04 17:11:54 -0700
 </div>
 </div>
 </body>

server_py/flatgov/static/docs/CRS_REPORTS.html

@@ -490,7 +490,7 @@ <h2 id="_collecting_the_data">Collecting the data</h2>
 </div>
 <div id="footer">
 <div id="footer-text">
-Last updated 2021-04-30 15:04:03 -0700
+Last updated 2021-05-04 17:11:54 -0700
 </div>
 </div>
 </body>

server_py/flatgov/static/docs/DATABASE.html

@@ -454,6 +454,7 @@ <h1>Setting up the FlatGov db for Django</h1>
 </ul>
 </li>
 <li><a href="#_create_db_migration">Create db migration</a></li>
+<li><a href="#_load_fixtures">Load fixtures</a></li>
 <li><a href="#_create_billsmeta_json_file_via_django_command">Create billsMeta.json file via Django command.</a></li>
 <li><a href="#_review_models_with_django_admin">Review Models with Django Admin</a></li>
 <li><a href="#_review_data_from_the_django_shell">Review data from the Django shell</a></li>
@@ -647,6 +648,20 @@ <h2 id="_create_db_migration">Create db migration</h2>
 </div>
 </div>
 <div class="sect1">
+<h2 id="_load_fixtures">Load fixtures</h2>
+<div class="sectionbody">
+<div class="paragraph">
+<p>The About page is created as a fixture. To load it, run</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre class="highlight"><code>cd server_py/flatgov
+python manage.py loaddata about_us.json</code></pre>
+</div>
+</div>
+</div>
+</div>
+<div class="sect1">
 <h2 id="_create_billsmeta_json_file_via_django_command">Create billsMeta.json file via Django command.</h2>
 <div class="sectionbody">
 <div class="ulist">
@@ -748,7 +763,7 @@ <h2 id="_review_data_from_the_django_shell">Review data from the Django shell</h
 </div>
 <div id="footer">
 <div id="footer-text">
-Last updated 2021-04-30 15:04:03 -0700
+Last updated 2021-05-04 17:11:54 -0700
 </div>
 </div>
 </body>

server_py/flatgov/static/docs/DATA_BACKGROUND.html

@@ -847,7 +847,7 @@ <h2 id="_committee_documents">Committee Documents</h2>
 </div>
 <div id="footer">
 <div id="footer-text">
-Last updated 2021-04-30 15:04:03 -0700
+Last updated 2021-05-04 17:11:54 -0700
 </div>
 </div>
 </body>

server_py/flatgov/static/docs/DEPLOYMENT.html

@@ -1157,7 +1157,7 @@ <h4 id="_provide_sudoer_access_only_to_those_who_need_it_caution">Provide sudoer
 </div>
 <div id="footer">
 <div id="footer-text">
-Last updated 2021-04-30 15:04:03 -0700
+Last updated 2021-05-04 17:11:54 -0700
 </div>
 </div>
 </body>

server_py/flatgov/static/docs/ES_SIMILARITY.html

@@ -664,7 +664,7 @@ <h3 id="_install_logstash">Install Logstash</h3>
 </div>
 <div id="footer">
 <div id="footer-text">
-Last updated 2021-04-30 15:04:03 -0700
+Last updated 2021-05-04 17:11:54 -0700
 </div>
 </div>
 </body>

server_py/flatgov/static/docs/README.html

@@ -908,7 +908,7 @@ <h2 id="_relevant_committee_documents">Relevant Committee Documents</h2>
 </div>
 <div id="footer">
 <div id="footer-text">
-Last updated 2021-04-30 15:04:03 -0700
+Last updated 2021-05-04 17:11:54 -0700
 </div>
 </div>
 </body>

server_py/flatgov/static/docs/SCRAPING.html

@@ -761,7 +761,7 @@ <h2 id="_relevant_committee_documents">Relevant Committee Documents</h2>
 </div>
 <div id="footer">
 <div id="footer-text">
-Last updated 2021-04-30 15:04:03 -0700
+Last updated 2021-05-04 17:11:54 -0700
 </div>
 </div>
 </body>

server_py/flatgov/static/docs/UPDATES_CELERY.html

@@ -984,7 +984,7 @@ <h4 id="_statements_of_administration_policy">Statements of Administration Polic
 </div>
 <div id="footer">
 <div id="footer-text">
-Last updated 2021-04-30 15:04:03 -0700
+Last updated 2021-05-04 17:11:54 -0700
 </div>
 </div>
 </body>

server_py/flatgov/static/docs/USCONGRESS_SCRAPER.html

@@ -545,7 +545,7 @@ <h3 id="_initialize_data_for_congresses_until_now">Initialize data for Congresse
 </div>
 <div id="footer">
 <div id="footer-text">
-Last updated 2021-04-30 15:04:03 -0700
+Last updated 2021-05-04 17:11:54 -0700
 </div>
 </div>
 </body>

server_py/flatgov/static/docs/index.html

@@ -644,7 +644,7 @@ <h2 id="_legislative_calendar">Legislative Calendar</h2>
 </div>
 <div id="footer">
 <div id="footer-text">
-Last updated 2021-04-30 15:04:03 -0700
+Last updated 2021-05-04 17:11:54 -0700
 </div>
 </div>
 </body>