Latent Space Podcast 6/1/23 [Summary] - Building the AI × UX Scenius — with Linus Lee of Notion AI

Original Link: Building the AI × UX Scenius — with Linus Lee of Notion AI

Summary

On the Road with Latent Space: A Visit to Notion New York

In a special episode of the Latent Space podcast, Alessio from Decibel Partners and Swyx from Latent Space, forgo their usual setting to interview Linus at the aesthetically consistent Notion New York headquarters. Linus reflects on Notion's distinctive design ethos which emanates from the company's founders, with a unique taste and focus.

Discussing Linus's diverse background, the trio delve into his journey from studying computer science at Berkeley to his prolific stint with multiple tech-based entities such as Replit, CatalystX, Hack Club, IdeaFlow, and finally, Notion. They touch upon Linus's Indiana roots, highlighting his fondness for small towns, despite now thriving in the urban buzz of New York.

A significant portion of the conversation centers on the AIUX SF meetup, a gathering that originated from a Twitter conversation and saw a confluence of machine learning and human-computer interaction enthusiasts. The meetup was an effort to bridge the gap between AI technology and intuitive user interfaces. Here, they reminisce about various innovations presented, especially those that moved beyond traditional auto-complete functions. These included Amelia's tool allowing users to alter text tone using brushes and Linus's prototype featuring semantic attribute sliders for text modification.

Towards the end, Alessio and Swyx touch upon Linus's penchant for side projects. Linus shares his philosophy for successful side hustling, emphasizing the importance of manageable scopes, using familiar tools, and consistently learning and iterating.

Throughout, the theme of innovation, the love for creating, and the joy of exploring new technological intersections are evident.

Exploring AIUX, Notion AI, and Linus's Sabbatical Journey into AI

Swyx and Alessio interview Linus about AI User Experience (AIUX) and its application, particularly referencing Notion AI. The key points are:

1. Knowledge Work & AI: Linus emphasizes that knowledge work is not just about content generation. It's also about understanding, synthesizing, proposing actions, and other knowledge-to-knowledge tasks. AI can greatly assist in these areas.

2. Potential of AI beyond Text Generation: While current uses of AI, like in Notion AI, are heavily geared towards text generation, its potential is much broader. Linus talks about using AI for knowledge synthesis and understanding, as well as for automation, like writing code or proposing actions.

3. Flexibility vs. Intuitiveness in Product Design: Alessio and Linus discuss the trade-offs in product design when using AI. While AI provides unparalleled flexibility, it can overwhelm users if not presented with some guardrails. A key design challenge is to guide users towards actions that AI is good at while also setting their expectations.

4. Novelty & Flexibility in AI Products: Alessio contemplates the current enthusiasm in "prompting" tools and wonders if, in a few years, most AI products will have a built-in intuitive path, thereby negating the need for users to prompt. Linus thinks that while guardrails and set pathways will likely become more prevalent, having a fully custom prompt or action can remain as an "escape hatch" for power users and for discovering new use cases.

5. Linus's Personal Journey: Linus shares his transition from web engineering to focusing on AI. He took a year off in 2022, partly as a sabbatical and partly to explore AI more deeply. This period gave him insights into the capabilities and potential of AI, which eventually led him to join Notion.

Swyx is keen on understanding Linus's personal journey and motivations, especially his decision to take a sabbatical to deeply dive into the realm of AI before joining Notion

The Evolution and Influence of Notation and Interface in the Digital Age

Swyx and Linus engage in a deep dialogue about the role of notation in various domains, from medicine to software engineering. Linus emphasizes the importance of custom notations in each discipline that elevates understanding beyond just raw words. This concept, dubbed "Notational Intelligence," highlights the significance of tailored notations allowing experts to work with higher-level concepts with ease.

Linus asserts that while text is a universal notation, its broad scope can be limiting in specialized domains, suggesting a need for advancements in this area. As the conversation segues into the realm of user interfaces (UI), Swyx underscores that while constraints can birth creativity, they alone are insufficient for invention. This leads Linus to share his experience at Notion, emphasizing the pivotal role of dominant market players in defining UI paradigms post the emergence of foundational technologies, such as personal computers and mobile devices.

Diving into the history of user experience, Swyx brings up the iPhone’s influence, hinting at how a small team's UI choices became the standard for modern smartphones. The conversation then touches upon the persistence of the QWERTY keyboard layout despite its lack of ergonomic efficiency and its origins in typewriters.

Finally, Linus offers a unique perspective on the evolution of buttons. From their mechanical origins, where they physically completed circuits, to today's digital world, where on-screen buttons merely simulate the older, physical version. This, Linus says, is a testament to the "cascade of conceptual backwards compatibility" in design that continues to shape our digital interactions.

The Evolution of Design and the Role of Agents in AI Workspaces.

Alessio initiates a conversation around the love-hate relationship with skeuomorphic design, reminiscing about early iPhone icons. Linus counters by drawing a comparison between design trends and fashion, suggesting skeuomorphism's recurring popularity. Linus goes on to elaborate on how skeuomorphism provides users with an intuitive mental model to engage with interfaces, such as iPhone volume controls.

Alessio shifts the discussion to "agents" and how their interfaces remain challenging. He highlights how interfaces like Calendly act as agents, albeit without real AI. Swyx points out how this approach has room for improvement, such as using AI agents for more personalized interactions.

The conversation moves on to the complexities in designing agent interfaces, with Linus emphasizing the balance between trust and control. Linus states that while it's vital for users to trust AI agents, it's equally crucial for users to control and constrain an agent's outputs. He adds that, particularly in collaboration platforms like Notion, it's essential to delineate AI-generated content from human-produced content.

Swyx and Linus further discuss the potential of agents and the challenges in managing their automated actions. They ponder over reversible changes and batching edits for user approval. Both agree on the parallels between human-AI and human-human collaborations, cautioning against solely anthropomorphizing AI.

Concluding, Swyx asks Linus to define Notion. Linus defines it as a "connected workspace," a hub for company docs, wikis, and workflows. He explains that the core strength of Notion lies in its "block" abstraction, where every element, be it a paragraph or page, is a block, emphasizing its fluidity and modular nature.

Notion AI: The Journey to Integrated Artificial Intelligence

Swyx inquires about Notion AI, seeing it as a startup within Notion itself, and wonders how committed Notion is to the AI wave.

Linus explains that Notion AI originated from an offsite hack weekend by Ivan and Simon. They were inspired by GPT-3 and the broader idea that software tools should be tailored to individual needs, leading to the creation of Notion AI. Notion envisions AI as a crucial part of its platform, just as databases and blocks are.

Alessio brings up the balance between user expectations and innovation, questioning how Notion reconciles the two.

Linus believes that having a vast user base is both a challenge and an advantage. Understanding the diverse uses of Notion helps them determine how to implement AI within it.

Swyx notes that many users struggle with creatively employing Notion AI, feeling restricted by the tool’s open-ended nature.

Linus admits that generality can be intimidating and acknowledges the challenge in striking a balance. He details the different AI facets within Notion: AI writer for content creation, AI block for automatically summarizing content, and AI autofill for databases.

Alessio questions how backlinks, a notable feature, fit into the AI evolution, considering its potential for interlinking documents and concepts.

Linus posits that, just as code files in programming can be seen as a graph of interconnected functions, text documents can be viewed similarly. A future challenge and opportunity lie in creating tools or AIs that can navigate and understand this complex network of knowledge.

Prompts, AI, and the Evolution of Interface Engineering

Swyx engages in a deep dive with Linus about prompt engineering at Notion, a relatively new yet rapidly evolving discipline.

• Prompt Engineering at Notion:

  • Notion uses prompts that are complex, particularly in the task of summarizing a wide range of potential content, from meeting notes to news articles.

  • Notion transitioned from instruction-following models to chat-based ones like Claude and ChatGPT Turbo, which made few-shot prompting easier.

  • Multilingual prompt support is critical for Notion. This involves handling numerous languages, ensuring the model provides output in the language the document is written in.

  • Prompt engineering also involves evaluation. The prompts need continuous updating based on feedback, and some prompts are extremely lengthy, reaching thousands of tokens.

  • For evaluation, Notion uses language models to judge other language models, questioning the bias in such models.

• Challenges & Future of Prompt Engineering:

  • There are concerns about long prompts taking up valuable context window for users.

  • There are methods to compress prompts and with advancements in AI, models can handle longer contexts, making this a temporary challenge.

• HCI and AI:

  • Historically, HCI (Human-Computer Interaction) and AI were seen as competitors, each aiming to make computers work better with humans through different approaches.

  • With AI's growing impact on user interfaces, there's a need for a standardized set of best practices, tools, and frameworks.

  • The concept of an AI engineer is emerging, bridging the gap between a software engineer and a machine learning researcher.

• Community Building:

  • Building a space where interested professionals can share ideas and collaborate is crucial. Past innovations, whether in physics or arts, have flourished when like-minded individuals came together.

  • The virtual space also offers an opportunity for collaboration beyond geographical boundaries, allowing talent from around the world to connect and innovate.

The Evolution of AI: Costs, Challenges, and the Future of Text Interaction

Rapid Progress & Cost Efficiency in AI During a lightning round of questions, Swyx and Linus discuss the surprising speed at which AI costs have decreased, especially evident in models like GPT-3.5 Turbo and DaVinci O3. Linus recalls how he tested the Notion AI autofill on a large database, and despite consuming millions of tokens, the test was cost-efficient, thanks to the affordability of new models. He jestingly mentions that with certain tests, the cost comes close to making Notion lose money, causing Alessio to joke about negative gross margins.

Challenges & Potential in AI Linus identifies one of the most pressing challenges in AI as building reliable and predictable systems, especially for automation. He points out that while AI can produce varied responses, achieving consistency in outputs is vital. To solve this, he suggests the co-generation of programs and having AI synthesize code for specific tasks, which then can be audited and improved upon by humans. He appreciates the confluence of AI with programming languages, compilers, and interpreters, and hopes the synergy between them will lead to better solutions. Swyx also highlights recent AI guidance solutions like LMQL and Microsoft's Guidance.

A Forward-looking Perspective on AI & Text Both Swyx and Linus agree that we're just at the beginning of the AI journey. Linus emphasizes that if humans are to remain for thousands of years more, it's unimaginable to think that our writing tools and systems won't evolve from their present state. He encourages a vision that looks beyond current conventions and expects disruptions, like the potential replacement of transformers in AI models in the years to come.

The conversation concludes with Alessio expressing gratitude for Linus's insights.