Filecoin Product: Where Slate Missed

[jimmylee]

Where Slate Missed

Some of us are excited about Slate:WWW, but there are valid problems with our Filecoin integration. I am hoping to address all of them here.

Glossary

Slate:WWW - The website that everyone uses today
Slate:Desktop - The desktop version of Slate that is offline & Lotus focused.

Was it your goal to be a Filecoin Client?

Yes! In the beginning Slate was intended to be a Filecoin Client GUI. In July 2020 our team name was actually the Filecoin GUI Team.

Over the months up until our launch in September 2020, we evolved something like this:

If you feel like Slate:WWW is a Filecoin Client, GUI, or product today, you may feel that Slate:WWW does not fail at this. However, I do want to address the other opinion that people hold about Slate:WWW.

Slate:WWW does not feel like a Filecoin product

In the social undercurrent of what is said about Slate:WWW, a common phrase I hear is "Slate is not a Filecoin Product".

When I try to understand this I recognize the following:

• (1) It is true that Slate:WWW has features that don't depend on the Filecoin Network in order for the product to function.
• (2) It is true that Slate:WWW does not support all of the features that the Lotus CLI supports.
• (3) It is true that Slate:WWW has a very low success rate of Filecoin storage deals.
• (4) It is true that Slate:WWW does not have Filecoin retrieval deals working.
• (5) Is is true that Slate:WWW requires centralized authentication to use the product at the moment.

I am aware of these, and I acknowledge this must change. If you agree that these points are legitimate problems with Slate:WWW, you should continue reading.

(1) — "Slate has features that don't depend on the Filecoin Network in order for the product to function."

When we started to integrate one to one with Textile's Hub (https://textile.io/) August 2020, the product deviated from the shape of an exclusive Filecoin Client to a multi-purpose web application that supports many user stories. The team believes that all user stories should eventually guide users to the Filecoin Network.

We want real people to learn to love Filecoin and experience the opportunities the Filecoin Network provides the internet community through Slate:WWW. Obviously this is not the case if our integration with the Filecoin Network isn't working properly.

So in the short term, we convinced many people to trust Slate:WWW as a place to keep their data. And we continue to perform this responsibility well thanks to the help of Textile. As a side note: Slate is made possible by Textile. As a team they are elite and phenomenal to work with. Thank you @andrewxhill and co. I mean it.

Shout out to @tarafanlin for such an awesome page.

However every single time collaborators talk about Slate:WWW, everyone seems to forget our intention.

• Real people store real data on Slate:WWW, and whenever a user passes 100 MB, we store their data on Filecoin if it is technically possible. We only fail because of miners!
• Every time Textile provides a Filecoin feature that works, we add it immediately to Slate:WWW and treat the feature like a first class citizen. I treat this as importantly as remembering to breathe.

This is not enough. Obviously our team must do more.

Action plan:

• TLDR: Filecoin is the destination, all paths lead to Filecoin.
• Slate:Desktop will be prioritized and will map to the original vision shared in How We Made Slate 11:00 before Textile Hub.
• Slate:Desktop will run against a Lotus node.
• Slate:Desktop will be offline first.
• Slate:WWW will continue prioritize and implement features from Textile above all other humanist design explorations.
• https://sentinel.slate.host will become more integrated into Slate:WWW.
• Learning from our peers at Digital-MOB: Slate:WWW will add a CID checker, block explorer, and storage oracle.
• Slate:WWW will add a miner database with reputation, and Slate:WWW storage deal history.
• Slate:WWW will add a retrieval marketplace (Bullet point: 4) that focuses on Slate:WWW user storage deals.

(2) — "Slate does not support all the feature that the Lotus CLI supports."

I feel bad about this. Why should we call ourselves a Filecoin Client if we don't support the same features the Lotus CLI supports?

The solution to this problem is simple, we should provide a GUI for each feature of the Lotus Client.

Lets bring this all to Slate:Desktop!

Action plan:

• Slate:Desktop is prioritized.
• Slate:Desktop's implementation will commit to parity with the Lotus CLI in its future GUI.

(3) — "Slate has a very low success rate of Filecoin storage deals."

This is the unfortunate event that spurred the initial disdain for Slate:WWW. I acknowledge Slate:WWW is a dead-end for people who want to use the Filecoin Network or even re-use our example code to learn how to develop on Filecoin. The problem is summarized in this excerpt:

The end result is that, even for our most advanced users like Textile, storage deal success rates are <25%. And this is aggregated across hundreds of users. This is untenable. For individual users, the story is even worse. Textile has had to tell Slate:WWW to stop making Filecoin storage deals because the volume of deals, given the unreliability of deals, means that deals get stuck in the pipeline for days/weeks.

Slate:WWW's infrastructure is capable of making 1200+ storage deals for unique real buckets with real data within an hour. Here is an example of the workers we run in parallel:

What went wrong:

• No one has a directory of reliable miners that can seal storage deal data and efficient rates. Textile is not an exception to this. The closest thing we have is this cross referenced list I produced: https://sentinel.slate.host/api/mapped-static-global-miners
• Textile found out that encrypted bucket storage deals do not work. Half of our deals are encrypted on behalf of the Slate:WWW users to respect privacy.
• Some miners are asking for too much FIL.
• Data transfer problems: Graphsync Transfer Failures lotus#4755 (thanks @whyrusleeping)
• Deals that were listed as successful in https://slate.host/api/v1/deals occasionally had miners that were no longer available. @ribasushi graciously provided miners to help the original Slingshot V1 contest get started but had to take them down. We need a method to repair miners: [https://slate.host] Repairing deals that are on miners that no longer exist. textileio/textile#389

Action plan:

• We will see if Marcus's Equinix data center miners can work reliably. I am going to shadow this work and learn as much as I can.
• I am working with @whyrusleeping to build a real miner so I understand how to make storage deals like the other experts we have.
• I will implement my own storage deal broker service so I understand the full stack of issues that involve multiple storage deals. Other people currently use Amazon S3 as a hot storage mechanism. I'll try something locally.
• I will work as closely as possible with @jsign and Textile to be as helpful as I can be.

(4) — "Slate does not have Filecoin retrieval deals working."

This problem is exacerbated by problem 3.

Slate:WWW's retrieval mechanism through Textile's Hub is still in development. Slate:WWW fails at retrieving deals from the Filecoin Network. Simply put, this must be fixed or we won't be able to uphold our promise to the Filecoin community.

Action plan:

• Slate:Desktop development is prioritized and will run its own Lotus node.
• Slate:Desktop will perform retrieval through a Lotus node directly and not through other infrastructure.
• I will work closely with @asutula and @carsonfarmer to have a working retrieval bucket story on Slate:WWW as soon as possible.

(5) — "Slate requires centralized authentication to use the product at the moment."

We wanted to introduce people to the world of Filecoin with as little friction as possible. So we introduced an authentication user experience that humans in the Web2 world are familiar with. With a username and password, anyone can create an account on Slate. Slate:WWW does not collect any user data and Slate:WWW does not track users. We felt like this was a great start, but Slate:WWW can and must do a lot better.

Everyone who feels like Slate is partially centralized like Keybase is correct in their assumption.

An example of the parity we have with other sister products.

Our team will make Slate:WWW feel much more decentralized, almost as fully decentralized as Slate:Desktop will be.

Action plan:

• Next version of Slate:WWW does not require creating an account.
• A visitor of Slate:WWW can provide a LibP2P token to use Slate:WWW. They can generate it themselves, or we can provide it for them.
• We will explore ways that browsers with identity (Beaker browser) can use Slate:WWW if they can find the authentication token in your access controlled storage.

Closing Notes

I hope we can renew the confidence that people had in Slate:WWW as a Filecoin Client. A renewed prioritization in Slate:Desktop will help ensure we are solving the problems stakeholders want us to solve.

• Please let me know if there is anything else I should address, all of my time is completely yours.
• We will announce our 2020 Q4, 2021 Q1, and 2021 Q2 plans soon. Feel free to make any suggestions to our roadmap. Our goal is to have a roadmap that is easy to follow and transparent with all of our users, collaborators, and Protocol Labs.

Thanks for the privilege to work on the Filecoin Team.