API: using private API endpoints on public listing pages

[agjohnson]

The issue here is that for public listing pages, unauthed users are using the project detail v3 API and that requires authentication. Because of this, the list of downloads and the documentation URL buttons are not working correctly. This was discovered in #152

We had a conversation around this already, and I believe we decided to make some of our detail API responses public instead of authorized. We might have to consider how this affects readthedocs.com, but this seems like the easiest fix.

The other option is to hardcode the version URL and version downloads in the template. These are expensive when listing a number of versions however, which is why these are dynamic calls in the first place.

[agjohnson]

Completely unauthenticated APIv3 access seems to be the problem here, the mixin ProjectQuerySetMixin used by the version API endpoint seems to allow public access for detail views, but still does assume the user is authenticated. In the case of any public project views, the visiting user is not going to be authenticated at all.

I'm not exactly sure what the API needs to allow public access like this, or what exactly we want to change to allow this. I think allowing this in the API makes sense, as the dashboard shows the same data publicly, but this obviously complicates the authorization check.

@humitos do you have opinions on what we do here with APIv3? Do we need to add a class to authentication_classes to allow unauthenticated API access on public projects or the community instance?

[agjohnson]

We discussed this more and it seems like it shouldn't be too hard to allow unauthed access to APIv3 or some subset of APIv3. The work looks to be primarily in tests.

Our initial concerns were performance and tracking users, but performance at least is less of a concern now. We don't yet have an idea of whether we do this change globally or just start with a limited number of views, similar to the notifications authentication changes.

[humitos]

Related issue/PR that performs a similar change on the notifications API endpoint to list notifications on public builds: readthedocs/readthedocs.org#11449 (comment)

[agjohnson]

The places where the dashboard is using API interactions on public views:

• Listing project's versions, on version and build listing.
• Listing user's projects, on the user public profile page. However, it probably makes sense to remove UI that requires public API access on this view instead of allowing project listing? The user profile public listing doesn't need the same UI elements as the authed project listing.
• Build detail view, on embedded popups
• Build detail and listing build's build commands, on build detail page. Note, the dashboard is currently stuck on APIv2 because of API: BuildCommands and missing fields to build v3 API readthedocs.org#11179 though
• Listing notifications (API V3: Allow other users to see build notifications from public projects readthedocs.org#11449)

I think that about covers things for now, but I might have more on this list.

[agjohnson]

Also noting, the work where I was blocked on this is

• Page responses times #128

So I think I can continue there and I'll disregard the side effect of public views not working with the API calls for now.

[stsewd]

With readthedocs/readthedocs.org#11485 anonymous access will be working on .org, on .com, our .api() querysets filter all resources by the organizations the user belongs to. So, anonymous access won't work there.
If we want to also allow anonymous access on .com, I can try doing a PR (hopefully it doesn't require lots of changes...)

[agjohnson]

Personally, I find public version/build listing views of projects to be an odd edge case to design around.

I get that historically it's been the case on community, but I do feel we could at least do away with these views on commercial (and all of the complexity conditionally public views brings) and it wouldn't be noticed by users. I think the same is mostly true for community, but there might be some users used to using the dashboard logged out at this point.

Instead of altering queries, and making this logic/query even more complex, I think we more easily could build a good failure mode into these listing UIs. We could then see if anyone complains about the links on commercial.

[agjohnson]

I believe we fixed this on community with a hack and commercial might need some more thought on if we need to provide this feature.

Make sure to summarize where we are and what we need in a discussion.

[stsewd]

The main problem here is that we are not allowing users to access public resources that are outside their organizations, this is using API V3, our dashboard does allow access to public resources.

Even if a user is logged in, if it doesn't belong to the organization of a project that is public, it can't access to that project using the API.

[humitos]

I understand there are two possible directions here:

1. make commercial to work in a similar way as community do (e.g. making .api() queryset to return public resources)
2. remove the "public" concept from projects on commercial and don't serve dashboard page on them

Am I correct? If so, what are the pros/cons of each of them?

[stsewd]

remove the "public" concept from projects on commercial and don't serve dashboard page on them

Don't think this is an option, we do have open source projects using .com that want their contributors to have access to the build pages, and I remember other clients expecting that as well (if I remember correctly, that was the reason we added support for public builds if the project and version are public).

The option here would be to just keep the API as is. But if our new dashboard uses API v3 in some views that are expected to be public, then we do need to change the API.

[agjohnson]

The dashboard is indeed already using APIv3 on public listing views. The dashboard should be using APIv3 more frequently, but I was forced to use APIv2 in a number of cases where we don't have feature parity.

[humitos]

I feel I'm back to the beginning now. Is there anything to discuss here then?

[stsewd]

Since we want to keep the behavior from the old dashboard, while keep using API v3 on the new dashboard, we are going to allow access to public resources on .com too.

We may want to pay a little more attention to any related information from resources we may be leaking (https://github.com/readthedocs/readthedocs-corporate/issues/1845 for example), we are leaking that information to users under the same organization now, but if we made resources public, anyone will be able to access those.

[ericholscher]

This is a link to see where we're using expandable fields in the frontend: https://github.com/search?q=repo%3Areadthedocs%2Fext-theme+expand+language%3AJavaScript&type=code&l=JavaScript