Secret #2: To manage capacity and organizational capacity, carefully prioritize use cases.
Few organizations have the capacity or capacity to develop every GenAI product they would like to build. According to BCG's recent cross-industry leadership survey, 62% of executives who expressed dissatisfaction with their organization's progress in AI and GenAI cited a talent and skills shortage.
Therefore, to make the most of the GenAI opportunity, it is important for companies to determine which use cases to focus on. Resource requirements should be driven by three considerations that focus on value, readiness, and complexity. To successfully scale a GenAI product, companies should be able to answer "yes" to the following questions:
- Value potential: Are the benefits of this solution enough to drive significant usage and adoption?
- Organizational preparation: Is our team ready, willing and able to build this solution? Does the team have leadership support?
- Complexity of the approach: Is the proposed approach something we can achieve and maintain?
Companies must fight the temptation to prioritize based on only one or two of these dimensions. By focusing solely on potential value, a company may take on more than it is prepared for in terms of technical complexity. By focusing solely on organizational readiness, a company may create products that do not have strong enough market demand. A company that focuses solely on complexity may conceptualize solutions that are valuable and feasible, but may lack the leadership support necessary to execute them.
The best companies use a balanced scorecard across these dimensions to focus their energy and resources in the right places. Once companies achieve some initial successes, they can typically expand on the skills and capabilities they have developed within their organization and also create increasingly sophisticated solutions.
For example, a financial reporting services provider created a retrieval augmented generation (RAG) solution to help answer free-text questions about the company's financial performance. It took the company nine months to develop this product, and only half that time to develop subsequent solutions that leveraged a similar approach.
Secret #3: To optimize systems integration, look for interoperability and adaptability.
It's easy to lose sight of the fact that a GenAI solution requires many components in addition to the GenAI model. The front-end, data, core business systems, infrastructure and security must be developed to function optimally, both now and in the future. Companies should start developing these other components from the beginning, focusing on two main considerations.
Interoperability. To support high-quality, secure data exchange, the components of a particular GenAI solution must be interoperable with the organization's existing technology stack and with its other GenAI products. This helps ensure that different solutions can "talk" to each other, allowing for consistent workflows.
Adaptability. Given the rapid pace of today's technology cycles, it is critical to ensure that solution components can be updated cost-effectively at scale. Therefore, companies must create components with standardized APIs and services that can be easily exchanged for updates when they become available. A standard solution reference architecture can then be used to guide compliance with enterprise architecture principles and standards.
Often, companies will need to make substantial changes to a solution. For example, when using an off-the-shelf model, it may be necessary to โretrofitโ or upgrade to a newer version of a base model when it becomes available. That's what many Chat GPT-3.5 users did when GPT-4 came out. In other cases, companies may need to rebuild their solution entirely, such as when new technologies or techniques emerge or a strategic change requires a different approach (for example, moving from a RAG solution to a refined model). Although these changes go beyond what we would normally consider โapplication maintenance,โ companies should treat them as part of the ongoing maintenance cycle for GenAI products.
Secret #4: To ensure robust data preparation and management, master these new techniques and mitigate new risks.
To create GenAI products that meaningfully leverage a company's data assets, organizations need to be data ready. Any data, whether structured or unstructured, must be clean, machine-readable, and protected from exposure risks. This means implementing preparation and management techniques specific to GenAI solutions.
Our survey of information service providers showed that data readiness increases with scaling experience: two-thirds of respondents with experience scaling GenAI products report high data readiness. In contrast, only a third of participants who lack scaling experience said they are ready to use data. (See Annex 3.)
