Optimizing Build and Test Times: Coda's Success with Aspect

Challenges Faced

After adopting Bazel, Coda had a powerful tool for scaling up the size of their TypeScript codebase, and spent some effort resolving cycles in the dependency graph to improve incrementality.

However, the build and test process on CircleCI were consistently exceeding 10 minutes on average, often stretching beyond 30 minutes. They were spending the equivalent of two software engineer salaries to cover the cloud costs required to run their build and test processes.

The prolonged build and test times were hindering Coda's development efficiency, causing delays in project timelines, and impacting overall productivity. The challenge was to find a solution that could significantly reduce these times and streamline their continuous integration process.

Selection Process

After careful consideration, Coda selected Aspect Workflows, developed by Aspect, as the solution that met their criteria for efficiency, scalability, and ease of integration. One of the key criteria was to remain on their existing CI system (CircleCI). The decision to use Workflows was influenced by their experience having worked at Google.

Implementation

The implementation process was a testament to the user-friendly nature of Aspect Workflows. Coda reported that adopting our product was "a pretty smooth transition." This speaks to the ease of onboarding and the effectiveness of Aspect Workflows in aligning with Coda's existing workflows.

Results and Benefits

The impact of implementing Aspect Workflows was substantial. After the implementation, Coda achieved not only a boost in development efficiency but also a remarkable cost savings in CI cloud spend.

Customer Case Study

Challenge

• Bazel build & test on CircleCI taking over 10 min average, often over 30 min.
• Cost too high: 2 SWE quiv.

Solution

• Aspect Workflows trial for one month.
• Improve stability and uptime, evaluate spot instances.
• “Pretty smooth transition” for developers.
• Regular improvement through monthly releases.

Results

• 10x faster no-op build from 11 min to 1 min
• 2-3x speedup of typical build & test
• 67% reduced compute cost (despite higher usage)

“We went from having significant limits in CI ad tools to where the limits are now just due to our code”
_{Neal Norwitz, Software Egineer}

The implementation of Aspect Workflows yielded tangible and impressive results for Coda. Their no-op fully-cached build, which previously took 11 minutes, was optimized to an impressive 1-minute duration.

The typical build and test cycle at Coda, representing a critical component of their development workflow, saw a remarkable improvement, now operating 2-3 times faster than before. This acceleration has significantly increased the team's agility and ability to iterate swiftly.

Notably, Coda achieved a remarkable 67% reduction in compute costs. This is despite the codebase growing during this period, so the workload increased. This cost efficiency demonstrates the effectiveness of Aspect Workflows in not only enhancing performance but also in optimizing resource utilization. This reduction in compute costs allows Coda to allocate resources more strategically, contributing to a more cost-effective and streamlined development process.

Notably, the 10x speedup in build and 2-3x faster tests exactly match the user testimonial presented in the Bazel 1.0 blog post.

Client's Perspective

• The optimization journey at Coda provides valuable insights into their historical and ongoing efforts to enhance their software development pipeline. In 2018/2019, the introduction of Bazel initiated a gradual process of improving build and test. By January 2022, about half of the code base could be compiled under Bazel.
• Additional targets and tests were added to address scaling issues. The team considered Bazel a natural fit based on their experiences with using it internally at Google. However, limited familiarity with JS/TS Bazel integrations presented a learning curve, and efforts were made part-time until a year ago.
• The contributor investigated alternatives to Bazel but found none that met Coda's specific needs. The team, not married to Bazel, remains open to exploration but faces challenges due to their existing setup with Webpack and plugins. Aspect plans to provide better off-ramps for Webpack that leverage Bazel’s ability to serve as the build orchestration.
• As long as Bazel and the TypeScript ecosystem provide sufficient value, Coda plans to continue using them. Ongoing evaluations will be crucial to ensuring the selected tool remains the best fit.