Schema descriptions should be shown in the database tree like source descriptions

[MarkMacArdle]

Describe the feature

In the source tree in the left sidebar if you click on a source directory you see it's a page for its description and the tables within it:

However if you're looking in the database tree and click on a schema you don't get the same behaviour. Nothing happens so the previously selected thing is still present in the main view.

I have some long schema descriptions that I really want to be visible to other users, but currently you can only see if them for a source. So I think expansion of this view would be a great benefit.

Describe alternatives you've considered

You could make all your clean datasets a source and then users could look there. This would be restrictive on the modeller though and non-dbt users may only be familiar with database view of things and not know/want to look at the sources (thinking business users who only look at docs occasionally).

Additional context

Non database specific

Who will this benefit?

• general docs website users
• any team/company early in adoption or where users have low familiarity with dbt so won't know to look at the sources tab to see schema level descriptions.

Are you interested in contributing this feature?

Yes! Have been looking through the code but am quite unfamiliar with web development so finding it hard. What I've understood so far is that what's in src/app/sources needs to get made for schemas too. Not sure how to do that.

[jtcohen6]

@MarkMacArdle Thanks for the write-up! To clarify, you want to define descriptions for your dbt-created schemas, is that right?

I have some long schema descriptions that I really want to be visible to other users

Where are you defining these today?

I think there's a blocker for this issue, which requires a change in dbt. Today, dbt lacks a mechanism for defining description properties for schemas—with the sole exception of sources—because all other objects (models, seeds, snapshots) with corresponding database representations are defined on their own terms, outside of any hierarchy that corresponds to schema/object database organization.

At the same time, there's definitely interest in doing this from members of the community. There are some related issues from users who want to document schemas and persist those descriptions in BigQuery: dbt-labs/dbt-core#1714, https://github.com/fishtown-analytics/dbt/issues/2951

I think there are a few directions this could go:

• A new schemas block, configurable in any *.yml file (aside from dbt_project.yml). How would this account for the fact that schema names vary according to the target and generate_schema_name macro implementations?
• A "special" docs block, similar to custom overviews, named in a specific way: e.g. {% docs __dbt_me__ %}

[MarkMacArdle]

Hi @jtcohen6. Currently we use dropbox papers (markdown files) to document things but are planning to switch to this docs website.

dbt lacks a mechanism for defining description properties for schemas—with the sole exception of sources

The schema descriptions I was thinking of using are those defined in the dbt_project.yml. I've tried adding schema descriptions into BigQuery with dbt and this has worked fine:

models:
  +persist_docs:
    relation: true
    columns: true

  dbt_bbm:  # our dbt project name
    policies_io_bordereaux:
      +schema: policies_io_bordereaux
      description: >
        Tables of written and paid transactions that will be sent to the underwriter
      +materialized: table

      intermediate:
        +schema: policies_io_bordereaux_intermediate
        description: >
          For breaking up the steps involved in the queries for making the bordereaux.
    
    etc

To include longer descriptions I had planned to switch to the doc markdown files (haven't checked you can include those in a dbt_project.yml but presume you can).

Reading the linked issues I see using these descriptions doesn't seem to have been considered before. Why is that?

[jtcohen6]

I don't believe the description you're setting in dbt_project.yml is actually doing anything. (It doesn't raise an error, either, because you can set whichever node configs you want in dbt_project.yml, and if they match one supported by dbt or the adapter you're using, they'll take effect.)

Today, description is only supported as a resource property in other .yml files; it's not supported as a node config, i.e. you can't set it in config() macros or dbt_project.yml; this is a distinction that we're hoping to reconcile before v1.0 (dbt-labs/dbt-core#2401).

That's ultimately the reason this isn't appearing on the docs site: it's not yet supported as a dbt feature.

However, you said:

adding schema descriptions into BigQuery with dbt

Are you referring to a custom process by which you're using dbt to update those datasets in BigQuery with those descriptions?

[MarkMacArdle]

Apologies I was mistaken! Nope we don't have a custom process, I must have just gotten confused and thought dataset descriptions were showing up in BigQuery, but I tried it there and they definitely aren't.

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue, or it will be closed in 7 days.