Knowledge Base Tooling - Request for proposals!

The Developer DAO Knowledge Base is currently in the form of markdown on a GitHub repo:
GitHub Resources Repo

Challenge

With our current GitHub repo, we have some of the following challenges:

  • As the repo grows with more and more resources, it will make it harder for people to find content that is relevant to them
  • Even though there is some structure, the repo may not successfully handle all incoming ā€œlearning pathsā€ (e.g. onboarding new devs, quick links for documentation, finding great tutorials about a particular subject)

Suggestion:

  • Building a list of resources (links to video, articles, tutorials, courses, websites, etc.) on a platform with a database was suggested so that we can have a comprehensive list of resources with structure, labels, etc.
  • This database would enable us to contribute, manipulate, and view (filter / sort) a comprehensive list of resources

Task

We are looking for proposals from members of the DAO on which tooling stack / platform we should move forward with, which will act as the DAOā€™s formal database for web3 resources, which will then be used to power the Knowledge Base.

Considerations:

  • There will likely be friction if we decide to switch platforms at a later date, which is why weā€™re asking for a stronger thought process (i.e. proposals) right at the start.
  • The selected platform will ultimately dictate the stack we will build the Knowledge Base on. For example, if the platform has an external / public-facing API, we have flexibility in building a curated front-end knowledge base. Alternatively, if the platform has an easy way of manipulating views and a good user experience, but lacks a strong API, we may build the knowledge base on the platformā€™s front-end product.

Requirements:

Not all requirements are mandatory. Some platforms will be more advantageous than others in various scenarios, but will have other drawbacks.

  • Flexible properties so that we can add filters / sorts
  • Performant at scale, even with thousands of database entries
  • External / Public API so that we can build a front-end on top of it, for curation / other reasons
  • View manipulation or output to GitHub repo
  • Export in case we need to switch platforms
  • Form input so that we can encourage better collaboration and potentially add process (reviews, contribution rewards, etc.)
  • Integration into contribution platform (TBD pending formalization of contribution mechanism - tokens, etc.)

Examples of potential options:

  1. Use AirTable as the database
  2. Use Notion as the database and knowledge base
  3. Build database on Supabase (or other custom integration)
  4. (Feel free to submit your own proposal)

Proposal Structure

Proposals can be submitted informally by anyone in the DAO. We ask that proper thought be given to the requirements above. You donā€™t need to follow the structure below, but you should present some good arguments about why your tool / platform will really help the knowledge base.

Example structure:

  1. Platform / tool description and overview
  2. Examples of successful KBs using this tool
  3. Architecture (i.e. how the KB ā€œstackā€ will be built on top of this tool)
  4. Onboarding process (i.e. how do we start using it)
  5. Requirements (i.e. how does the tool handle the requirements above)
  6. Other notes

Current Proposals

Next Steps:

Any member of Developer DAO can submit a proposal in this thread.

Deadline

  • Please reply to this thread with your proposals.
  • Weā€™d like all proposals to be submitted by: Wed. Oct. 20 at 17:00 UTC.

Voting

  • Weā€™d like as many members as possible to view the proposals and submit their vote for the winning platform / tool.
  • Voting mechanism TBD.
  • Weā€™d like all votes to be submitted by: Fri. Oct. 22 at 17:00 UTC.

Happy to answer any questions, talk about new ideas, or partially-baked proposals on #knowledge-base in Discord.

7 Likes

How/where does one vote? :grinning_face_with_smiling_eyes:

PS: Just saw its TBD. Aight

1 Like

Not to bikeshed, but it might be worth changing the name of this resource - knowledgebase usually implies internal institutional knowledge, which we will need in the future. This tooling is more for a resource database or list, rather than a ā€œtrueā€ knowledgebase. This will probably influence voting.

1 Like

Airtable Knowledge Base

Platform

Airtable describes itself as ā€œa low-code platform for building collaborative appsā€. Imagine a relational database controlled through a very intuitive UI.

Itā€™s other tagline is ā€œThe power of a database with the familiarity of a spreadsheet.ā€œ

It also provides a number of extras which could be useful for building a knowledge base such as:

  • Different Views (grid/list) that can be embedded on web pages
  • Forms
  • DX friendly auto-generated API with docs

Examples

DAOists - DAOists.xyz Resource Base (Public) - Google Docs

Architecture

Airtable works via bases, tables and views.

Bases represent one entire dataset or database
Tables represent tables within a base
Views are various ways of viewing the data in a table

I propose the following architecture:

One main base

The following Tables:

  • Resource
  • Author
  • Glossary
  • Contributor

Here is a quick implementation of how this could work:

Resources - Airtable - Grid view

Authors - Airtable - Grid view

Glossary - Airtable - Grid view

The tables in this base would be populated by forms using Airtables native forms feature.

How we use it

Usage is locked to members of Developer DAo. We create a contributions doc similar to DAOists which shows how people can contribute and pin this in the knowledge-base channel.

We also then engage the community to encourage each other to add contributions. This should be easier by making it clear contributions will be recognised and rewarded.

There are 4 potential workflows for adding to the knowledge base.

Adding resources

If this is the first time someone is contributing

Fill out the Contributor form

Add a new resources to the list from an existing author

Fill out the Resources form

Add a new resource to the list from a new author

Fill out the Author form and then fill out the resources form

Add a new term to the glossary

Fill out the Glossary form

Tracking contributions

The Contributor table is used to track contributors to the knowledge base resources and glossary tables.

When a D_D member wants to contribute to the knowledge base for the first time, they need to add themselves as a Contributor by entering their discord handle (NOT their server nickname) into the Contributor form.

Moving forward they can then select themselves as the Contributor in the Resources and Glossary forms to track which content they have contributed to the knowledge base.

We can then see a count of their contributions to each using whatever filters we like i.e. last week/month/quarter. This will be useful for recognising and rewarding contributions moving forward.

We will also track who is adding authors for quality control although I donā€™t think adding an author should count towards contributions and thus potentially any reward.

Requirements

Flexible properties so that we can add filters / sorts

This provides many different built in features for how we can soft and filter using any categorisation we include in the base. For example, category, author, media (aticle/video etc.)/

Performant at scale, even with thousands of database entries

Airtable is used in production in many companies and there are plenty of stories out there of people building entire businesses using Airtable as their database. The current limit is 100,000 records per base on the highest plan.

We donā€™t have to manage anything.

External / Public API so that we can build a front-end on top of it, for curation / other reasons

Airtable auto-generates an API + documentation for every base, here is the one for this test implementation - Sign in - Airtable

Their JavaScript wrapper for their API is very easy to use and comes with rate limiting etc. built in. See Airtable.js

View manipulation or output to GitHub repo

We can host views of the dataset either as standalone pages or embedded as iframes, as either a list or a grid view.

We could then use their API to auto-generate the Markdown repo on GitHub.

Export in case we need to switch platforms

Iā€™m less knowledgeable in this area BUT you can export the tables as csv files so I assume we can then jus port this over to any relational database without to much trouble.

Form input so that we can encourage better collaboration and potentially add processes (reviews, contribution rewards, etc.)

Built in forms above.

Integration into a contribution platform (TBD pending formalization of contribution mechanism - tokens, etc.)

Have included a mechanism for tracking contributors in test implementation above.

Notes

Cost

Airtableā€™s free tier only includes 1200 records per base. This will likely become a limitation quickly if we have many people adding resources into the knowledge base.

The first paid tier is $10/seat per month and includes 5000 records

The second paid tier is $20/set per month and includes 50,000

I imagine weā€™ll break the free tier quite quickly however imagine it would take a while to break the first paid tier and even when we manage that Iā€™m not sure weā€™d ever break the second paid tier.

One other thing to think about here is the GB limit for attachments. It could be nice to include images for each resources time so we can display them in a more aesthetically pleasing grid view, unsure if this limitation affects our ability to do that.

Curation

Curation could happen once a week by filtering based on new resources/authors/glossary items added in the last week. This should be far easier than handling PRā€™s and issues on GitHub.

One thing to not re cost is that once we enter the paid ties each contributor would need a login or theyā€™d need to share a common one.

Auto-generating github repo

We should be able to do this via either the GitHub API and a node.js script (serverless function probably) OR GitHub Actions, although I donā€™t know the latter very well.

Using API to power other UIā€™s

V 0.1 could just host a view of the tables on the website in an iframe. We could then start working on a custom UI on top of the API for the website (or anywhere else) if thatā€™s what we wanted to do

4 Likes

All great points from @kempsterrrr and looking forward to this KB proposal.

Also in favor of Airtable. This weekend tested out both Airtable + Notion and found Airtable integration for a front-end the easiest to implement. Notion required a bit more time and since it is a newer API implementation, would rather use one that has been developed and documented for a while.

Had a look at all the forms set up and looks great. A potential suggestion is to have a single separate form that gets reviewed during the curation process to simplify it into one process.

3 Likes

Obligatory ā€œwe can build it ourselvesā€ point of view:

DAO SupaKnowledgebase

Supabase.io

In their own words:

The Open Source Firebase Alternative
Create a backend in less than 2 minutes. Start your project with a Postgres Database, Authentication, instant APIs, realtime subscriptions and Storage.
Serverless functions coming soon

  1. Magic (postgrest + some other stuff) Postgres database and API

  2. Front-end unopinionated with javascript SDK for data access

  3. Examples for all major frontend frameworks

  4. Supabase auth with magic links or username and password or 3rd party:
    a. Apple
    c. Azure
    d. Bitbucket
    e. Discord
    f. Facebook
    g. Github
    h. Gitlab
    i. Google
    j. Twitter
    k. Twitch
    l. twilio

  5. It is possible to write our own login but it is unclear how much work that would be (for ETH logins)

Examples of other KBs

Searching for knowledgebase turned up no results for KB built on Supabase, so we would be doing this on our own.

Architecture


The client(s) can be unique or shared applications with same or different frontends

Onboarding

  1. Get Supabase credentials and determine how secret keys will be controlled
    a. This seems like a large problem but solvable
  2. In parallel, individuals can work up what the various clients and pages will look like and the supporting SQL schemas we intend to use.
  3. Any user with valid credentials can interact directly with the database (see the concern with keeping the secret keys safe) so we could start populating without a frontend, or it can be done via the supabase web application.
    a. Building a management interface for the KB will make this easier and more accessible.
  4. Public facing web application also has to be created to make this useful to the public.

NOTE: Developers can run a local instance of supabase to create applications even if they have already tapped out their personal quota of projects on supabase.

Requirements

  1. Flexibility

    a. Itā€™s a Postgres database, all the flexibility of SQL

  2. Performance

    a. No specific benchmarking available on site, but also no blog posts about how Supabase sabotaged Postgres in their application. Draw your own conclusions, ultimately itā€™s a Postgres database behind an API server.

  3. API

    a. That is what this is built to provide
    b. A production ready Postgrest (postgrest.org) setup with auth, etc

  4. View manipulation

    a. Custom Frontend(s)
    b. Not sure about ability to support scheduled export to GH repo

  5. Export

    a. Can export the database content
    b. Frontend would be entirely our code (other than swapping out the supabase SDK for whatever the new connection would be)

  6. Form Input

    a. Custom content / forms

  7. Integration to Contribution Platform

    a. Custom forms
    b. Custom anything as a means of filtering (ethers to check tokens or signatures or the like)

Other notes

Upsides

  1. Complete customization possible
  2. Postgres database is easy to transfer to another platform
  3. We know a number of motivated web developers
  4. Supabase console allows for direct SQL and visual manipulation of the tables and data
  5. Repeatable pattern that other DAOs could potentially use

Downsides

  1. Requires build of all interfaces other than psql connections and the Supabase admin panel

    a. Supabase has made some aspects open source ā†’ can use those if desired (depending on feature set)

  2. No existing web3 auth for supabase

  3. Requires initial database design and probably some forward looking thought to make sure we donā€™t sabotage future changes

    a. We will be starting with a small enough dataset that we will be able to perform pretty severe restructuring without getting into the dangerous territory for which DBAs get paid.

A vote for building it ourselves is a vote for not being (as) constrained by another platform - but weā€™ll still be bound by Supabaseā€™s decisions unless we decide to eject from there (or an equivalent database provider)

5 Likes

@kempsterrrr I was on the call for the team building the new DAOists website yesterday and theyā€™ve cobbled together something functional using Airtable (which we knew) and Softr:

Theyā€™ll probably have something public by next week that allows visitors to search and filter their Airtable DB from a front-end made with Softr.

Itā€™s not a long-term solution, but a pretty functional/awesome quick win if we do go with Airtable. Not sure where else to mention this or if you want to fold it into your proposal.

1 Like

This is dope @kaxline - Iā€™ve pretty much modelled this proposal on DAOists approach. Theyā€™re executing really well, although Iā€™d mention they clearly did a fair bit of planning before launching :joy: and have quite experienced DAOists (Bankless, FWB etc.) in their midst already.

My thoughts re website are alluded to here:

Totally with you re using the toolset that allows to have the biggest impact the fastest. That said, my thoughts are we can probably roll our own solution for hosting it on the website (assuming we want to do that) relatively quickly once the website team are given clearer direction by ratifying P-2: Defining our Mission, Values, and Goals and How we move forward as a DAO - Season 0. Plus the team have already put a lot of effort into the site.

1 Like

@kempsterrrr OK great! I should also mention that the main developer wasnā€™t too happy with Softr as a tool and they are planning to replace it with a custom site ASAP. Just depends on if we want to launch ~3 weeks earlier, or however much time it would save us.

1 Like

Notion Knowledge Base Proposal

Platform / tool overview

Notion is an application that provides components such as notes, databases, kanban boards, wikis, calendars and reminders.

Users can connect these components to create their own systems for knowledge management, note taking, data management, project management, among others.

Main differentiations:

  • Easy to get started and start writing content thatā€™s more curated for users
  • Externally friendly UI thatā€™s customizable
  • Database (Table) is very flexible, has customizable views, filters, and sort functions, and supports formulas, relationships to other databases, etc
  • Database items can be turned into pages and support additional content
  • Real-time collaboration
  • Free to start (pricing below)

Challenges:

  • API is still in Beta
  • Limitations to UI / Design
  • Limitations to views on database
  • Lack of native forms (need to use a third-party service to achieve forms, but TypeForm seems like a very good integration)
  • Lack of automation / integrations (but will be expanded soon as API is in Beta)

Interesting Ideas:

TL;DR:

  • Notion is very good at creating a Version 1.0 of a Knowledge Base.
  • Notionā€™s pages act as a good way of creating customizable views with mixed content (e.g. commentary by the DAO, curated content, etc.).
  • Since itā€™s very user friendly and supports a very flexible database system, itā€™s easy to build an external facing Knowledge Base.
  • It will be challenging to build custom integrations and create our own views outside of Notion with our own capabilities (e.g. Wallet authentication, etc.)

Example created:

  • Example of D_D Knowledge Base on Notion
  • Resources table is shown with some properties listed as an example.
    • Author is a soft reference to ā€œPeopleā€, but actual relations can be used.
    • Views can be created with specific filters / sorts
  • Pages can be created however you like and can have specific views of databases (which might help with curation). See ā€œLearn about Web3ā€ in the Beginners section

Examples in the wild

Extremely large set of templates available from Notion here:
https://www.notion.so/Notion-Template-Gallery-181e961aeb5c4ee6915307c0dfd5156d

Notionā€™s own Help / Support / Documentation:
https://www.notion.so/Help-Support-Documentation-e040febf70a94950b8620e6f00005004

Examples using super.so for theming:

Architecture

Example: Notion ā€“ The all-in-one workspace for your notes, tasks, wikis, and databases.

Notion would be a collection of:

  • Pages
  • Databases (Tables)

Pages would be created as the primary ā€œviewā€ for everything in the Knowledge Base. Pages would be created for the following purposes:

  • Landing Pages
  • Sections per skill level (Beginner, Intermediate, Advanced)
  • Sections per technology (Ethereum, Solana, Polkadot, Polygon, etc)
  • Sections per content medium (Video, Articles, Interactive Games / Tutorials, etc.)
  • Marketing pages to boost DAO member content
  • FAQs

Databases would be created for the following:
(Stole this from @kempsterrrr but it makes total sense)

  • Resources
  • People (Content Creators, etc.)
  • Glossary
  • Contributor
  • Frequently Asked Questions

To surface a subset of the database into a user-friendly view, a specific filtered view can be created on any page.

  • Example: ā€œLearn about Web3ā€ Page can reference all resources containing the tag ā€œBeginnerā€ and ā€œConceptualā€

Forms would be used on top for members of the DAO to add new content (publicly available, so they do not need Notion accounts)

Onboarding process

  1. Initial Setup
  • DAO Core Team sets up KB in Notion
  • KB Team sets up KB overall structure, pages, databases, permissions
  • KB Team sets up form tool (3rd party)
  1. Contribution
  • Members of the DAO can add new resources to the form tool
  • Members of the DAO can propose changes via comments on each page

Requirements (i.e. how does the tool handle the requirements above)

Flexible properties so that we can add filters / sorts

  • Available

Performant at scale, even with thousands of database entries

  • No limit on databases
  • Some mentions of slow down on large databases
  • Database limits on view (25, 50, or 100 at a time)
  • Searching is generally slow, because it searches pages and other content

External / Public API so that we can build a front-end on top of it, for curation / other reasons

  • Public API is in Beta
  • (Have not researched how easy this is to work with)

View manipulation or output to GitHub repo

  • Currently none supported

Export in case we need to switch platforms

  • Export to CSV

Form input so that we can encourage better collaboration and potentially add process (reviews, contribution rewards, etc.)

Integration into contribution platform (TBD pending formalization of contribution mechanism - tokens, etc.)

  • TBD

Other notes

Pricing

  • https://www.notion.so/pricing
  • Free tier available
    • Unlimited blocks
    • Access to API (Beta)
  • Will likely need to upgrade to $8/seat/month for team collaboration / unlimited team members
4 Likes

The SOFTR approach seems exceptionally limited based on what @javier123454321 has been running into there. I think we (D_D) can manage a custom website pretty fast (particularly if itā€™s just the frontend presenting data to non-contributors).

1 Like

Thanks again for all the proposals.

Iā€™ve created a poll for everyone to vote.

  • Please note that your vote will be visible by everyone
  • Voting is for members of Developer DAO ONLY
  • Voting is open right now!
  • Voting will close Fri. Oct. 22 at 1500 UTC
    (Poll will still be open, but weā€™ll collect the results by then)

Actions

  • Before you vote, please ensure you understand the proposals above
  • Vote on the tool / proposal
  • (Optional) Post your vote thoughts / opinions on this thread, if it helps provide more insight

Summary / Examples:

A quick summary and links to examples are provided below, but more details are in the actual proposals.

Airtable

  • Leverage Airtable for powerful and flexible database with built-in forms
  • Build out front-end for curation purposes
  • Example of Resources Table: Airtable - Grid view
  • Example of Resources Form: Form

Supabase

  • Build database (via Supabase) and custom front-end
  • Manage resources through Supabase dashboard or create front-end to interact with data

Notion

Strapi (Unofficial proposal)

  • Strapi was proposed by @Raz during our KB weekly sync
  • A CMS could be a great way for us to manage content in our tables as well as content for the website with a custom front-end
  • @wolovim has initiated a Strapi live demo (valid for 24 hours).
  • Vote for this option if you think this could be the better option. We would have to do more exploration to validate whether this would be a great fit for us.

Poll

Vote on the Knowledge Base Tool
  • Airtable
  • Supabase + Custom Front-end
  • Notion
  • Strapi
  • Other

0 voters

If other, please reply to this thread with more details.

Thanks everyone!

2 Likes

Thank you to everyone who voted! The polls have closed.

Notion: 18 votes
Airtable: 15 votes
Total: 33 votes

Thank you to everyone who participated!
Special thank you to @kempsterrrr @gjsyme for submitting proposals!

Next steps will be to set up the structure, process, tooling for Notion.

4 Likes

Good tool to know, Softr and super and ā€¦