ProgrammingPro #72: Optimizing Builds with CMake & ccache, Android 15 Developer Tools, PyPi Security Risks, and JDK 23 Updates
Welcome to this week’s edition of ProgrammingPro!
In today’s Expert Insight, we bring you an excerpt from the recently published book, CMake Best Practices -Second Edition, which discusses how to optimize and maintain CMake projects using a compiler cache, specifically ccache, to speed up C and C++ builds.
Related Titles from Packt
Get to grips with CMake and take your C++ development skills to enterprise standards
Use hands-on exercises and self-assessment questions to lock-in your learning
Understand how to build in an array of quality checks and tests for robust code
Get the eBook for $39.99 $27.98
Get the Print Book for $49.99 $39.98
Comprehensive introduction to C++ programming and detailed explanations of algorithmic concepts.
Step-by-step setup of development environments, with practical exercises and real-world examples.
In-depth coverage of control structures, and visual aids such as trace tables and flowcharts.
News Highlights: Android 15 launches with enhanced developer tools; PyPi faces security risks in package management; .NET 9 introduces crucial security updates, .NET Framework remains vulnerable; and Elasticsearch reverts to open source amid tech shifts.
My top 5 picks from today’s learning resources:
But there’s more, so dive right in.
Stay Awesome!
Divya Anne Selvaraj
Editor-in-Chief
PS: This month’s survey is now live. Do take the opportunity to tell us what you think of ProgrammingPro, request learning resources, and earn your one Packt Credit for this month.
Developing for iOS? Setapp's 2024 report on the state of the iOS market in the EU is a must-see
How do users in the EU find apps? What's the main source of information about new apps? Would users install your app from a third-party app marketplace?
Set yourself up for success with these and more valuable marketing insights in Setapp Mobile's report, "iOS Market Insights for EU."
🗞️News and Analysis🔎
Android 15 is now officially available: The version introduces a range of developer tools in Android Studio and Jetpack including new PdfRenderer API capabilities, new OpenJDK APIs, new SQLite APIs, and new Canvas drawing capabilities.
TypeScript 5.6 enters release candidate stage: Changes include renamed types, reverted search behavior for tsconfig.json files, and improvements from the prior beta phase.
PyPi repository vulnerable to deleted package hijack, say researchers: Researchers at JFrog have identified a vulnerability in the PyPi package repository where attackers can hijack deleted package names and upload malicious versions.
Microsoft makes breaking change in .NET 9 for security, leaves .NET Framework insecure: Microsoft is removing the BinaryFormatter class from .NET 9 due to its unfixable security risks, although it will remain in the .NET Framework.
Elasticsearch will be open source again as CTO declares changed landscape: This move comes after the projects shifted from the Apache 2.0 license to more restrictive terms in 2021, aimed at preventing companies like AWS from offering these products as managed services without collaboration.
Uno Platform 5.3 Released: The version Introduces official support for JetBrains Rider, enhances the Hot Reload experience across all supported IDEs, and includes new UI controls.
.NET Aspire 8.2: Components Renamed to Integrations, Enhanced Testing, and More Improvements: The version also introduces a Hosting package to simplify resource management in local development and enhanced setup.
🎓Tutorials and Learning Resources💡
Sponsored Mini Course on AI worth $399 for free: 200+ hours of research on AI-led career growth strategies & hacks packed in 3 hours. This course on AI has been taken by 1 Million people across the globe. Register Now.
Sponsored Live Webinar - The Power of Data Storytelling in Driving Business Decisions: Join to learn about Data Storytelling and turn complex information into actionable insights for faster decision-making. Register for free.
Python
Web Scraping With Scrapy and MongoDB: Guides you through setting up a Scrapy project, building a web scraper, extracting data, and storing it in MongoDB. Read to also learn about testing and debugging techniques.
For more Python resources, go to PythonPro
C# and .NET
Lesser-known operators in C#: Details unique logical and null operators alongside functional ones, which enhance code expressiveness and error handling.
🎥.NET and C# are in trouble. Here is what I'd do: Delves into the debate surrounding the rapid evolution of .NET and C#, featuring community reactions and proposals for enhancing adoption and support.
🎥🎓Tutorial | Let's Build Our Own ArrayPool - Deep .NET with Stephen & Scott: Explores the intricacies of object pooling, optimizing array length, and managing memory pressure, alongside concurrent queue operations and thread management.
C and C++
Qt/.NET — Using QML in a .NET WPF application: Discusses the integration of QML into .NET WPF applications through the proposed Qt/.NET toolkit, allowing seamless interoperability between C++ and .NET.
Honey, I shrunk {fmt}: bringing binary size to 14k and ditching the C++ runtime: Covers strategies including using type erasure to minimize template bloat and applying the Dragonbox algorithm for efficient numeric formatting.
HashMap in 25 lines of C: Demonstrates how to construct a simple hash table in C, accommodating any value that C understands.
Java
🎓Tutorial | Static classes and inner classes in Java: Shows how nested classes can organize code and explains static member classes. Read to gain insights into using nested classes in Java to maintain cleaner code structure.
JDK 23 and JDK 24 - What We Know So Far: Provides a breakdown of new Java Enhancement Proposals (JEPs) across different categories and updates on major Java projects like Amber, Loom, Panama, and Valhalla.
Hide and Seek with Collections in Java: Delves into the Collection Accessor Method pattern using Eclipse Collections, analyzing various strategies to manage relationships within object-oriented domains in Java.
JavaScript and TypeScript
Poor man's signals: Explains how signals can be implemented using JavaScript's
EventTargetclass, providing a simplified, event-driven approach to track changes to data.Compilation of JavaScript to Wasm, Part 3 - Partial Evaluation: Explores the use of the first Futamura projection to transform interpreter code into compiled WebAssembly code, highlighting the practical challenges and solutions.
Advanced React Component Design with TypeScript: Focuses on creating reusable, flexible, and type-safe components while enhancing developer experience (DX).
Go
Exploring Goja - A Golang JavaScript Runtime: Explores how Goja, a JavaScript runtime for Golang, effectively simplifies embedding JavaScript in Go applications, particularly for tasks involving complex data manipulation.
Telemetry in Go 1.23 and beyond: Discusses the introduction of telemetry in Go 1.23, allowing developers to opt-in to sharing toolchain usage data to improve bug fixes, prevent regressions, and enhance decision-making.
Rust
Unmaking the Makefiles - exorcising the Rust compiler test suite one file at a time: Narrates the journey of rewriting 352 Makefiles into Rust using the run-make-support crate, which was part of a Google Summer of Code project.
Async Rust can be a pleasure to work with (without `Send + Sync + 'static`): Explains how to simplify async Rust development by avoiding
Send + Sync + 'staticbounds through Structured Concurrency and thread-per-core.
Swift
Cross-Platform Swift - Building a Swift app for the browser: Discusses the development of Swift applications for non-Apple platforms, specifically focusing on creating a simple Swift app that runs in a browser using WebAssembly.
Swift Enums 🤝 Design Systems: Elaborates on the advantages of using enums to define UI components, providing a clean and maintainable way to encapsulate branding elements like colors and icons into reusable components.
PHP
🎓Tutorial | PHP and Web Dev Phobia: Guides you through setting up a virtual machine using Vagrant and VirtualBox, installing PHP and Lighttpd on Debian, and creating a simple PHP page.
SQL
Best practices for prompt engineering with Meta Llama 3 for Text-to-SQL use cases: Outlines the process of setting up the model with Amazon SageMaker JumpStart, using structured prompts, and leveraging model versions for optimized dialogue and SQL parsing.
Ruby
Write a Ruby C Extension to Use CLIPS from Ruby - Part 3: Focuses on enabling Ruby methods to be called from CLIPS rules, showcasing how to manipulate and interact with Ruby objects and data types within the CLIPS environment.
Transpiling Go & Java to Ruby using GPT-4o & Claude 3.5 Sonnet: Discusses the extension of the DevQualityEval LLM code generation benchmark to include Ruby and utilizing LLMs like Claude 3.5 Sonnet.
Kotlin
🎓Tutorial | Data Analytics With Kotlin Notebooks, DataFrame, and Kandy: Guides beginners on how to set up and use Kotlin Notebook to analyze and visualize data, specifically using a weather data CSV file as an example.
🌟Best Practices and Advice🚀
Efficient DevSecOps Workflows With a Little Help From AI: Explores how AI aids in identifying security vulnerabilities, offering explanations and fixes, and streamlining operations with features like root cause analysis.
Glue and coprocessor architectures: Highlights how systems can efficiently handle complex computations by separating them into highly specialized tasks performed by coprocessors and more general, flexible operations.
Gemma explained - RecurrentGemma architecture: Discusses how the architecture employs a hybrid model called Griffin that combines gated linear recurrences with local sliding window attention for efficient long context processing.
Lean Architecture: Proposes a 'Lean Architecture,' focusing on efficiency, minimal design waste, and maintaining software with an adaptable, problem-solving approach rather than following strict architectural guidelines.
🧠Expert Insight📚
Here’s an excerpt from “Chapter 14: Optimizing and Maintaining CMake Projects” in the book, CMake Best Practices -Second Edition, by Dominik Berner and Mustafa Kemal Gilor, published in August 2024.
Using a compiler cache (ccache) to speed up rebuilds
Ccaches work by caching compilations and detecting when the same compilation is done again. At the time of writing this book, the most popular
program for caching compile results is ccache , which is open source and distributed under the Lesser General Public License 3 (LGPL 3). The ccache program not only affects incremental builds but also fresh builds, as long as the cache is not deleted between the two runs. The cache created is portable between systems running the same compilers and can be stored in remote databases so that multiple developers may access the same cache. Officially, ccache supports GCC, Clang, and Nvidia CUDA Compiler (NVCC), but people claim to have run it for MSVC and Intel compilers. When using ccachewith CMake, it works best with Makefile and Ninja generators. At the time of writing this book, Visual Studio was not supported.
To use ccache with CMake, the CMAKE_<LANG>_COMPILER_LAUNCHER cache variable is used, where <LANG> is replaced with the respective programming language. The recommended way is to pass this in using a preset, but to enable ccache for C and C++ inside a CMakeLists.txt file, the following code can be used:
find_program(CCACHE_PROGRAM ccache)
if(CCACHE_PROGRAM)
set(CMAKE_C_COMPILER_LAUNCHER ${CCACHE_PROGRAM})
set(CMAKE_CXX_COMPILER_LAUNCHER ${CCACHE_PROGRAM})
endif()
Passing the variable from a preset or from the command line or a preset is also a good alternative, especially because the configuration of ccache is done easiest by using environment variables.
Using ccache with the default configuration might already bring a considerable improvement regarding build times, but if the build is a bit more complex, further configuration might be necessary. To configure ccache, certain environment variables starting with CCACHE_ can be used; for full documentation of all configuration options, refer to the ccachedocumentation. Common scenarios that need special attention are combining ccache with precompiled headers, managing dependencies that are included using FetchContent, and combining ccache with other compiler wrappers, such as distcc or icecc for distributed builds. For these scenarios, the following environment variables are used:
To work efficiently with precompiled headers, set
CCACHE_SLOPPINESStopch_defines,time_macros. The reason for this is thatccachecannot detect changes in#definesin the precompiled header, and it cannot tell if__TIME__,__DATE__, or__TIMESTAMP__are used when creating precompiled headers. Optionally, settinginclude_file_mtimetoCCACHE_SLOPPINESSmight further increase the cache hit performance, but it carries a very small risk of a race condition.When including big dependencies that are built from source (for instance, by using
FetchContent), settingCCACHE_BASEDIRtoCMAKE_BINARY_DIRmight increase the cache hit rate; this might bring a performance boost especially if there are many (sub)projects fetching the same dependency. On the other hand, if the sources in the project itself are the ones that take more time to compile, setting this toCMAKE_SOURCE_DIRmight bring better results. It needs to be tried out to learn which one brings the better result.To work with other compiler wrappers, the
CCACHE_PREFIXenvironment variable is used to add commands for these wrappers. It is recommended to useccachefirst when chaining multiple wrappers so that the results of other wrappers may also be cached.
Passing environment variables to CMake using a configure preset, as described in "Chapter 9, Creating Reproducible Build Environments", is the recommended way; this can either be combined with detecting ccache inside the CMakeLists.txt file or the ccache command may also be passed using the following preset:
{
"name" : "ccache-env",
...
"environment": {
"CCACHE_BASEDIR" : "${sourceDir}",
"CCACHE_SLOPPINESS" : "pch_defines,time_macros"
}
},
With these configurations, using ccache can yield very large benefits to the compile time, but caching compiler results is a complicated matter, so to get the full benefit, the ccache documentation should be consulted. In most cases, using ccache will probably bring the most performance benefit with a relatively trivial setup. Other tools, such as distcc for distributed builds, work very similarly from the CMake perspective, but require a bit more setup work.
CMake Best Practices -Second Edition was published in August 2024. Packt library subscribers can continue reading the entire book for free or you can buy the book here!
Get the eBook for $33.99 $22.99
Get the Print Book for $41.99
🛠️Useful Tools⚒️
Daytona: An open-source tool that simplifies the setup of development environments, allowing developers to configure and launch their workspaces with a single command across various platforms and architectures.
Sparo: An open-source tool developed by TikTok engineers to enhance the performance of large monorepos by utilizing sparse checkout and partial clone features of Git for more efficient operations.
lmnr: an open-source observability and analytics platform tailored for complex LLM applications, integrating automatic telemetry, analytics, and prompt chain management with a focus on performance, transparency, and scalability.
That’s all for today.
We have an entire range of newsletters with focused content for tech pros. Subscribe to the ones you find the most useful here. Complete ProgrammingPro archives can be found here. Complete PythonPro archives are here.
If your company is interested in reaching an audience of developers, software engineers, and tech decision makers, you may want to advertise with us.
If you have any suggestions or feedback, or would like us to find you a Programming learning resource on a particular subject, take the survey or leave a comment below!