Shape Up for AI

Shape Up is a project management framework that emphasizes iterative delivery and lightweight upfront shaping rather than big design up front. The core principles of timeboxed cycles, cross-functional team collaboration, and flexibility can be adapted to guide AI development while allowing for rapid experimentation and learning. According to McKinsey: open, rapid learning, coupled with continuous feedback loops create the leading AI organizations.

Generative AI holds enormous potential to augment human creativity and productivity. However, as these models become more powerful, it is crucial that innovation progresses hand-in-hand with ethical responsibility.

This article outlines key principles and best practices to develop generative AI features safely and beneficially based on real-world needs by using the Shape Up Method applied to AI projects.

By taking a measured approach grounded in transparency and risk mitigation, teams can steer clear of pitfalls and realize the full upside of this technology.

Introduction to Shape Up Methodology

First – if you're new here and to Shape Up – here's a refresher. The Shape Up method was created by Basecamp co-founder Ryan Singer as a way to improve project planning and execution.

Unlike traditional waterfall development, Shape Up applies agile principles at the portfolio level to help companies build the right things in the right order.

At its core, Shape Up consists of three repeating phases: Shaping, Betting, and Building.

• Shaping – In the Shaping phase, companies identify promising opportunities and define projects, setting the constraints but leaving details flexible.
• Betting – The Betting phase involves pitching these shaped project ideas and deciding which ones to pursue based on business priorities.
• Building – Finally, the Building phase executes on the selected projects in discrete, time-boxed cycles called scopes.

The key principles behind Shape Up include focusing on the customer's current reality, embracing uncertainty, deciding by rough consensus, and working in small, integrated teams. Proponents argue that Shape Up enables organizations to quickly align around priorities and deliver value without getting bogged down in requirements and estimations early on.

Shape Up aims to counter issues that can plague traditional project management like endless analysis, vague deliverables, and poor synchronization across teams. For modern software development, and especially AI projects, Shape Up offers an intriguing lightweight framework to keep teams nimble, collaborative, and customer-focused.

Shape Up AI: Key Principles for AI Projects

When embarking on AI development, it's crucial to establish sound foundational principles to guide the effort. Adopting responsible, ethical approaches enables the creation of models that provide value to users and society.

This section outlines core recommended principles to keep top of mind across the stages of an AI project. From clearly defining the purpose early on to responsibly monitoring outputs, these actionable guidelines support development of fair, transparent, and effective AI systems.

Whether crafting a small prototype or an expansive production model, reflecting on these principles helps shape AI for good. By taking the time to internalize this ethos, teams can build trust with users and deploy AI responsibly.

Applying Shape Up AI is easy, but it's important to remember these principles when creating generative AI apps and features.

Clearly Define the Purpose

Generative AI can produce wide-ranging outputs, so it is crucial to have a well-defined use case and goal guiding the effort. This provides direction for data collection, model development, and evaluation.

Curate High-Quality, Diverse Training Data

Models learn from their training data, so invest significant time collecting diverse, representative datasets without biases. Prioritize high-quality examples that exemplify the intended purpose.

Start Small, Iterate Quickly

Don't try to build an enormous generative model right off the bat. Begin experimenting with smaller models to validate capabilities and refine the approach through rapid iteration.

Monitor Outputs for Harmful Content

Test generative models thoroughly before release to catch any problematic, biased or malicious outputs. Have a plan to handle these issues responsibly.

Benchmark Performance with Metrics and Human Evaluation

Use quantitative metrics and qualitative human assessments to evaluate model improvements. Balance relevance, accuracy, coherence and other goals.

Design for Transparency

Enable users to recognize generative content and provide explanations to establish appropriate trust. Do not present AI outputs as organic.

Mitigate Risks Through Responsible Development

Consider potential misuses early and incorporate strategies to reduce harms. Responsible development is crucial for deploying generative AI.

Listen to User Feedback

Frequently gather user reactions and preferences to steer the direction of generative features. Incorporate insights from a diverse range of perspectives.

By following these principles, teams can develop generative AI that is both innovative and ethically grounded. Focusing on real-world benefits while mitigating risks leads to responsible progress.

Accelerating Shape Up with AI Orchestration

While the Shape Up methodology provides a strong framework, executing on AI projects can still be challenging. AI development involves unique workflows like data wrangling, model prototyping, training loop optimization, and monitoring model drift.

Orchestrating these complex tasks efficiently requires purpose-built tools.

AI orchestration platforms can streamline technical complexity and accelerate development cycles. When integrated with Shape Up, these platforms enhance the methodology's effectiveness:

The right AI orchestration platform can accelerate Shape Up through capabilities like:

• Low-code environment for rapid prototyping during shaping
• Pre-built components to assemble solutions faster
• Tools for creating interactive demos when pitching ideas
• Integrated platform to develop, deploy, and monitor apps
• Reusable model and data catalogs to enable iteration
• Workflow automation and MLOps for efficient delivery
• Performance monitoring to identify improvement opportunities

Seek an agile platform that facilitates prototyping concepts, assembling solutions, developing iteratively, and gathering user insights.

With the right AI orchestration tools, teams can shape up quality, high-impact AI applications faster.

Shaping Phase: Defining the Project and Constraints

The shaping phase is a critical part of the Shape Up process. This involves clearly defining the problem to be solved and setting the right constraints before jumping into solutions.

For AI projects, problem definition requires careful analysis of the customer need, the current process, and the desired outcomes. The goal is to shape a project around delivering concrete value to users, not just implementing technology for its own sake.

Some key aspects of shaping AI projects include:

• Understand – Understanding the customer workflow and pain points. Where do users struggle today? What takes the most time or causes frustration? Sharply defining this helps shape the solution.
• Identify –  Identifying the ideal user experience. How should the perfect AI solution behave? Painting this picture separates must-haves from nice-to-haves.
• Analyze – Analyzing existing data and infrastructure. What data sources are available? What integrations are needed? This informs the practical constraints.
• Consider – Considering risks and unknowns. What could go wrong when applying AI? What expertise is required? Highlighting risks upfront prevents issues later.
• Stakeholder Alignment – Balancing stakeholder requests and technical constraints. Stakeholders may ask for the moon, while engineers ground requests in reality. Shaping strikes the right balance.

The shaping process brings all stakeholders to the same page by co-creating concrete project scopes. For AI, scoping the minimum viable product avoids over-engineering. This phase sets the team up for success in the next bet and build stages.

Betting Phase: Selecting the Most Promising Ideas

Once the shaping work is done, the next phase is betting. This involves deciding which project ideas are worth investing in for the upcoming build cycle.

The rhythm of innovation at our company is marked by the energy and anticipation of betting. This is when product managers pitch promising new AI project ideas to company leadership. With enthusiasm, they outline the customer problem to be solved, proposed generative solution, estimated timeline, and resources required. The room is filled with a sense of possibility as they describe how machine learning could enable new capabilities.

After each pitch, a spirited betting table discussion follows. Participants ask probing questions about the technical approach, request clarifications, and debate the merits and risks of funding each AI project. Product managers highlight how their proposals align with company goals and strategy.

Based on the pitches and collaborative debate, leadership decides which moonshot projects to bet on and approve for the build phase. The goal is to pick ideas with the highest potential value based on customer and business impact. The criteria weighs factors like feasibility, resources required, data considerations, and technology unknowns. AI projects often require special consideration around training data needs and unproven techniques.

Teams feel empowered during betting because they get to shape projects based on their expertise. Leadership bets on people as much as ideas. Winning bets are where the business impact justifies the effort required. Lower-value proposals may be shaped further and reconsidered in a future cycle. With the bets placed, the builders get to work making the AI magic happen.

Building Phase: Executing the Project and Delivering Results

The building phase is when the rubber meets the road. This is where the team executes the project that leadership bet on, working to deliver concrete results.

Once the bet is made, it's time to build. The project takes shape as the team divides efforts into independent scopes - manageable chunks of work that can be iterated on in sprints. Engineers focus intently on one scope at a time, avoiding inefficient context switching between tasks. Each scope has a clear objective, like creating a minimum viable AI feature. Defining achievable milestones maintains momentum.

Fluidity reigns during the build phase. There are no prescribed roles, enabling team members to contribute flexibly wherever they can add value. Problems get solved collaboratively in real-time, not through roadblocks and status reports. Progress is measured tangibly via working software, not theoretical advancement against a master plan.

Every 6 weeks the team demonstrates the latest software, collecting stakeholder and user feedback to steer ongoing development. For AI projects, this cycle allows for continuous learning about the technology capabilities and evolving business goals.

With the build phase balancing structure and flexibility, the team can deliver incremental value efficiently while minimizing wasted effort. Through focus, collaboration, and user input, they bring the AI product vision to life.

Some key aspects of the build phase for AI projects:

• Scope Hammering – Projects are broken into scopes - independent pieces that can be built and tested. This divides work into iterative sprints.
• Focus – Engineers focus on one scope at a time without multitasking. This avoids wasting time context switching.
• Clear Goal – Each scope has an objective, like creating a minimum viable AI feature. Achievable scopes maintain momentum.
• Team Work – There are no prescribed roles. Team members contribute wherever they can add value.
• Real-time Contributions – Problems are addressed collaboratively in real-time, not through roadblocks and STATUS reports.
• Working Demos – The team measures progress via tangible working software, not theoretical progress against a plan.
• Feedback – At the end of each 6 week cycle, they demonstrate the software to stakeholders and users to collect feedback.

The building phase balances structured execution with flexibility. For AI projects, it allows for rapid iteration and continuous learning about the technology and business goals.

Done right, the team can deliver value efficiently while minimizing wasted effort.

Real-World Examples and Lessons Learned

The Shape Up methodology can be applied to a wide range of AI projects, including natural language processing systems. For example, a social media company used Shape Up to develop an AI classifier for detecting policy violations in user content.

During shaping, the product and data science teams interviewed content moderators and defined the core problem as inconsistent accuracy and slow moderation throughput. The solution pitched and selected was a GPT-4 based classifier fine-tuned on the company's moderation data.

The build phase involved iterative training of the model with a hill chart tracking accuracy improvements every week. The team ran into challenges with biased model outputs, which led to changes in the training data composition. After 6 cycles, the classifier reached the accuracy targets.

Key lessons from this project included:

• Aligning the model capabilities with realistic business goals through clear problem shaping. This prevented overpromising on accuracy.
• Using hill charts to track model accuracy as the key metric instead of just training progress. This provided better visibility.
• Adapting the 6-week cycles to be shorter for faster iteration on model tuning.
• Monitoring model bias and adjusting training data and methodology accordingly.

The GPT-4 classifier increased moderation efficiency by 40%, outperforming previous models. Examining such real-world applications provides valuable insights on tailoring Shape Up specifically for AI development.

The Benefits of Shape Up AI

The Shape Up method provides a structured framework for effectively managing AI projects. By dedicating time to shaping the problem, selecting the right bets, and building in iterative cycles, teams can improve the focus and delivery of AI products.

Shape Up enables cross-functional collaboration, transparent progress tracking, and the flexibility to incorporate learnings throughout development.

However, the methodology must be adapted to the unique aspects of AI projects. The shaping phase should involve substantial upfront research and experimentation to define feasible problems.

The build cycles need to accommodate the prototyping and iteration inherent in model development. Metrics like accuracy and fairness are vital for hill charts instead of just output volume.

Overall, Shape Up promotes the kind of problem-driven, iterative approach necessary for AI development. The techniques help maintain alignment on the end goal amid complex ML engineering work.

Further incorporation of responsible AI practices like bias evaluation into the framework will be important.

With the right adaptations, Shape Up can become an essential project methodology for delivering impactful, ethical AI systems.