In August, we announced the Full-stack Development Pool vendors for the Agile Delivery Services Marketplace. As we continue to work on the alpha version of this marketplace, we’re beginning to plan out additional “micro-markets” as well. By micro-market, we’re referring to a cohesive grouping of technology-related products or services that are aligned to the targeted needs of both 18F and other government agencies.

Given our current mode of thinking, our approach to developing future micro-markets will share a common set of characteristics.

Lean startup principles

For each new marketplace that we create, we’ll continue to use a lean-startup approach: an initial “alpha” version (accessible only to or through 18F) for us — and vendors — to gain experience, followed by a “beta” version (potentially accessible to all of government) that incorporates lessons learned from the alpha phase. And further iterations, as necessary.

We can say with great confidence that we got a number of things right with the Agile Delivery Services Marketplace thus far, but there’s also an equal number of things we’d definitely do differently. Things we couldn’t possibly have predicted, no matter how many months of planning or preparation. We believe designing learning cycles into the development of our marketplaces will produce better results in areas such as the quality of vendors.

Creative evaluation methodologies

We’ll continue to explore new ways of evaluating vendors based on two core principles: (1) demonstrated capabilities over written narrative and (2) minimally sufficient evaluation criteria over completely exhaustive evaluation criteria.

The second principle requires a little explanation. If, for example, it only takes three out of 100 questions to determine whether or not someone can actually do a job, there’s no point in asking the other 97. To do so puts an unnecessary cost burden on both vendors and the government. We’re very sensitive to keeping the bid and proposal expense on vendors as low as possible. By doing so, we’re avoiding a situation in which vendors have to pass that additional cost onto the government in the form of higher prices. Not only that, they can invest more in hiring people who actually do the work (for example, engineers), as opposed to those who write about it, which the first core principle encourages as well.

Contract vehicle strategy

Regardless of how simple or well-thought-out designing a contract vehicle is, it’s still a lot of work and has the potential to duplicate something else that already exists in the federal government. Not only that, it creates a lot of extra work for vendors who have to bid on yet-another vehicle for which they already qualified for at another agency. For each marketplace, we’ll do our homework to ensure we’re not unnecessarily duplicating what’s out there.

Industry engagement

Our formula for engaging industry during the pre-award stage of the Agile Delivery Services Marketplace was highly effective and useful, for both vendors and the government. We made several positive adjustments to our acquisition strategy (for example, creating a separate award pool for design companies) based on interactions with industry.

We’ll replicate the same process for engaging industry, to include the following steps: new marketplace announcement, request for information, one or more industry days, pre-solicitation conference, solicitation, question and answer, and awards.

Where appropriate, we’ll create separate marketplaces for categories that can easily be divided into sub-categories. For example, data management or cybersecurity can naturally be divided into at least two sub-categories, such as software products and professional services.

Often, the government needs to use a combination of products and services together to solve a business or technical problem (for example, software to detect when a network device is down and a human to resolve the incident). From an acquisition standpoint, however,

they require different methodologies and expertise to evaluate and manage. Separate marketplaces also allow us to handpick which individual products and services “go on the shelf” as a way to assure complementariness and quality, as opposed to leaving those decisions to a reseller or systems integrator.

Bottom line, we’ll treat related products and services as related, but separate contracting activities.

Not just for the federal government

Since announcing the Agile Delivery Services Marketplace, we’ve received a number of inquiries from state and local governments expressing interest in using this marketplace. As 18F has come to learn, GSA has a long history of providing acquisition support services to customers at all levels of government, as permitted by law. We’ll look to build on this tradition by designing future marketplaces to be accessible to them as well.

Tooling to support marketplace activities

Our ability to develop, manage, and scale high-quality marketplaces — and to create delightful contracting experiences — requires proper tooling. Currently, we’re in the process of developing an electronic form for agencies to initiate and execute interagency agreements with 18F so they can work with 18F more efficiently. Shortly, we’ll begin work on another tool to streamline the question-and-answer process between vendors and the government during a solicitation. But it won’t end there. More tools will be needed, and we’ll build them.

Lowering barriers to entry

Every company who meets the minimum qualifications for doing business with the government should be given the opportunity to do so in the most frictionless way practicable. This is why we’re partnering with the GSA’s Federal Acquisition Service, and specifically the Office of Integrated Technology Services, to make it easier for vendors to onboard onto government-wide contract vehicles like IT Schedule 70. Recently, GSA released an RFI (now closed) to better understand these barriers. We’ve also piloted a fast lane process for onboarding. In addition, we’re working closely with the FedRAMP program to even further streamline the process for assessing and accrediting the security of cloud-based products.

Cross-functional teams

One of the big reasons agile works is because it emphasizes the use of cross-functional teams. When a team possesses comprehensive domain knowledge and the breadth and depth of skills needed to complete a project, several things happen: shared vision, fluid communications, creative problem solving, cross-training, and steady work progress (skillset redundancy helps to avoid resource capacity bottlenecks). This gives any project, not just software development, a much greater chance of success.

We’ll create our marketplaces no differently — using cross-functional teams with expertise in law, acquisition, technology, etc.

Value-added services

Many of the marketplaces will require new ways of buying (for example, modular contracting). To support agencies in making this transition, we will provide a variety of value-added services from hands-on acquisition and technical consulting to training workshops.

Openness and transparency

Openness and transparency are two of the foundational operating principles that pervade everything 18F does, from the code we write to how we deliver. There are so many benefits for doing so — cost savings, reusability, knowledge sharing, public accountability, etc. Where legally permissible, we’ll design our marketplaces to be equally open and transparent, from the code vendors write to the quality of that code to how well they’re performing on contracts.

Vendor performance science

Healthy competitive dynamics drive companies to perform at higher and higher levels. Openness and transparency will help foster this competition, but to push it even further, we will develop systematic and data-influenced ways to understand which vendors consistently perform above or below “the bar” based on a variety generic and marketplace-specific performance attributes. Those companies who are above the bar will continue to earn more opportunities for work, while those who aren’t will earn fewer — and may be competitively replaced with new marketplace entrants using onboarding and offboarding mechanisms.

Ultimately, understanding and influencing the factors that contribute to healthy competitive dynamics will better equip us to create and maintain high-performing marketplaces.

Planned marketplaces

Without further ado, here are the marketplaces that are currently on our radar, in no particular sequence and without specific timetables due to a host of planning uncertainties at this early stage in the game:

Horizontal marketplaces (support the specific needs of a wide-range of government buyers across different mission areas such as justice, defense, healthcare, homeland security, and finance/economics):

  • Infrastructure/platform (products, services)
  • Data management (products, services)
  • Developer productivity tools (products)
  • Cybersecurity (products, services)

Vertical marketplaces (support the specialized needs of a specific group of government buyers):

  • FinTech (products, services)
  • Health IT (products, services)

We’re truly excited about the opportunities that lie ahead for the vendor community and the government. Stay tuned for information about the ongoing development of the 18F marketplaces.