ProgrammingPro #40: Property-Based Testing, Docker's Build Cloud, Oracle's 2024 Java Vision, and Full-Stack Python
Bite-sized actionable content, practical tutorials, and resources for programmers
Welcome to a brand new issue of ProgrammingPro!
In today’s Expert Insight we bring you an exclusive case study from the book, Soar with Haskell, on property-based testing. Worth diving into, if you work in an industry that prioritizes complex algorithms and data analysis or on tasks requiring high reliability and mathematical precision.
News Highlights: Docker's Build Cloud streamlines local development; how does AI influence engineering quality?; Oracle's 2024 Java vision revealed; and open source secures concessions in the EU cyber law.
My top 5 picks from today’s learning resources:
But there’s more, so dive right in.
Stay Awesome!
Divya Anne Selvaraj
Editor-in-Chief
PS: If you have any food for thought, feedback, or would like us to find you a specific Programming learning resource for our next issue, take the survey! As a thank you, we will send you one free Packt credit to buy a book of your choice.
Advance your knowledge in Tech!
Dive into a world of endless knowledge with Packt’s 7-day FREE trial!
Access our entire library of 7500+ eBooks and Videos.
🗞️News and Analysis🔎
Lukewarm reception for Microsoft's Copilot Pro amid performance, cost grumbles: While some see the AI technology's potential, others are skeptical, citing concerns over the platform's effectiveness and the high price tag of $30 per user per month. Read for insights into what Copilot Pro offers and adoption challenges.
Docker introduces Build Cloud for accelerated local development: The new Build Cloud service accelerates local development by using cloud resources and a shared build cache, significantly reducing build times. Read to learn about Build Cloud’s functionality, benefits, and integration with existing development workflows.
Enterprises struggle with Agile methodology, reports long-standing survey of practitioners: The 17th State of Agile report reveals that Agile methodology is facing challenges in medium and large organizations, with only 44% of respondents finding it somewhat or very effective. Read for more insights.
The impact of AI use by developers on quality engineering: This article highlights the risks and opportunities AI presents in coding, debugging, and quality assurance. Read for insights into the need for quality engineers to understand and adapt to AI advancements.
Oracle’s plans for Java in 2024: Oracle plans significant advancements for Java in 2024, focusing on projects like Amber, Babylon, and Valhalla in the OpenJDK. Read to learn about Oracle's future vision for Java and gain insights into upcoming features and improvements that could impact your development work.
Open source wins concessions in new EU cyber law: The final text introduces an "open source steward" concept, protecting entities like the Python Software Foundation (PSF) from legal responsibilities for security issues in products using their code. Read to learn about how this change impacts you.
Brave Search now answers coding queries: Brave Search has introduced CodeLLM, an AI feature that provides high-quality answers to coding queries. Read to learn more about how the new feature enhances your search experience for coding-related queries with AI-powered, privacy-centric solutions.
🎓Tutorials and Learning Resources💡
Python
🎓Tutorial | Create a Web Interface for your LLM in Python: This tutorial demonstrates creating a chatbot web interface using the Taipy Python library and HuggingFace's API with the google/flan-t5-xxl model. Read for insights into web interface creation, LLM integration, and cloud deployment.
Python as a configuration language: Python allows rich, expressive syntax for defining configuration data, but is recommended not to be made importable. Read to discover alternatives, and the value of tools like Pydantic and CUElang.
🧰Full-Stack Toolbox: Python Edition: This article presents a selection of Python libraries tailored for various aspects of full-stack development, including web application, backend, and comprehensive frameworks. Read to enhance your full-stack development capabilities.
Java
MavenGate - a supply chain attack method for Java and Android applications: The attack exploits vulnerabilities in Maven-based technologies, including Gradle, by hijacking abandoned library dependencies. Read to learn more.
Java StringTemplates and logging, or why f-strings are not enough: This innovation will provide powerful, efficient, and safe logging APIs, without redundant string construction. Read to learn how you can improve logging practices in Java applications.
🎓Tutorial | Java mocks: A guide to mocking in Java: This comprehensive guide covers the setup of Mockito and Unlogged IDE plugin, for creating, managing, and utilizing mock objects and methods. Read for practical examples and best practices.
Go
💼Case Study | From slow to SIMD - A Go optimization story: This article details the optimization of a critical dot product function in Go, essential for Sourcegraph's Code AI tool, Cody. Read for insights into enhancing the efficiency of critical functions in system-level programming.
Exploring Go 1.22 - Effective HTTP Routing Strategies: This article examines the new HTTP routing capabilities in Go 1.22. Read for insights into the enhanced pattern matching for the server multiplexer in the standard library.
PHP
10 tools that every PHP developer should use: Tools covered include PhpStorm, a sophisticated IDE; Codeium, an AI-powered coding assistant; DataGrip, for database management; and PHPStan, for error detection. Read to discover resources to optimize your development process.
Building Maintainable PHP Applications - Framework Decoupling vs Framework Coupling: Framework coupling, where developers heavily rely on the framework for structure and functionality, can lead to difficulties in upgrades and inflexible code. Read to learn about the advantages of framework decoupling in PHP.
SQL
Mastering Maintenance - Strategies for seamless auto-updates on Azure SQL Managed Instance: This article details the Safe Deployment Practice (SDP) framework, maintenance windows options (Default, Weekday, Weekend), and the importance of retry logic in applications. Read to learn about the intricacies of maintaining Azure SQL Managed Instances.
🎓Tutorial | Writing a Lexical Analyzer for SQL in C#: This article explores lexical analysis, emphasizing its significance in compiling source code into tokens, with a specific focus on SQL. Read to gain insights into tokenization, finite automata, and pattern recognition in programming languages.
Game Dev.
🎓Free Access Course | Pro C++: This full fledged, free, online, gaming focused programming course is for those ready to dabble in advanced concepts. The site also offers an introductory level course.
🎓Tutorial | How to Build a Bouncing Ball Game using PyGame: This tutorial includes setting up the development environment, defining game concepts, initializing the app, creating game screens, and the main game loop. Read to gain hands-on experience in developing a simple yet engaging Bouncing Ball game using Python and PyGame.
C/C++/C#
🎓Tutorial | How to implement a custom object mapper in C#: While AutoMapper is useful for mapping objects with similar properties, it falls short in complex architectures or when dealing with incompatible types. Read to learn how to handle complex data structures.
The Formatting Library in C++20: This article distinguishes between compile-time (e.g., std::format) and run-time format strings (e.g., std::vformat), explaining their syntax and usage. Read to prepare to tackle more advanced topics like chrono and custom type formatting.
You Don't Need Enums in C#: This video discusses the use of enums in C# and advocates for a paradigm shift from using enums to representing characteristics or abstractions of objects. Read to learn an alternative method in C# to represent object characteristics.
Sharing C code between Swift and Kotlin for iPhone and Android apps: This article explains how the Skip transpiler, as of version 0.7.44, generates a gradle project for cross-compiling C projects into Android apps, supporting four native Android architectures. Read to learn how to efficiently integrate C code into Android applications.
JavaScript and TypeScript
💼Case Study | Reversing and Tooling a Signed Request Hash in Obfuscated JavaScript: This article offers a real-world application of reverse engineering and debugging techniques in the context of web security. Read if you are interested in bug bounty programs or work with web application security testing.
Top 5 JavaScript Tools for AI Engineering: This article talks about TensorFlow.js, AI.JSX by Fixie, ConvNetJS, Brain.js, and Tabnine. Read for insights into development, training, and deployment of AI models within JavaScript environments.
TypeScript Index Signatures – an in-depth look: This article provides an understanding of index signatures, their applications, and solutions to common related errors. Read to enhance your coding efficiency and problem-solving skills.
.NET
🎓Tutorial | Prepare an application for authentication: This article will guide you through creating an ASP.NET Core Web App, securing it with a self-signed certificate, and configuring authentication using Microsoft Entra. Read to learn how to securely set up and authenticate an ASP.NET Core Web App.
🎓Tutorial | Secure ASP.NET Core server-side Blazor apps: This article covers interactive server-side rendering, managing authentication state, and integrating external providers for additional claims and tokens. Read to learn how to secure your Blazor apps.
Ruby
🎓Tutorial | Numeric operations on value objects in Ruby: This Ruby tutorial demonstrates implementing a Value class that allows arithmetic operations with numbers, such as 2 + Value.new(3). Read to learn how to extend the functionality of a custom class.
🎓Tutorial | Behavior Driven Development in Ruby with RSpec: RSpec is a Ruby library which focuses on testing the behavior rather than the implementation of code. Read to gain skills in writing clear, behavior-focused tests.
Rust
Rust Data Types: This article covers Rust's basic data types with examples. Read to learn about Rust's comprehensive type system, enabling efficient and type-safe coding, crucial for robust application development.
Swift
COW2LLVM: The isKnownUniquelyReferenced Deep-Dive: This article delves into Swift's copy-on-write (CoW) optimization, a key feature for iOS engineers. Read to gain an in-depth understanding of this feature and its implementation in the Swift compiler.
Kotlin
Micro-optimizations in Kotlin: This article discusses micro-optimizations in Kotlin's Int.sign function, highlighting the need for both macro and micro-level optimizations in libraries like Jetpack Compose. Read to learn how to perform more efficient assembly-level implementations.
Microservices
Exploring the Horizon of Microservices With KubeMQ's New Control Center: This article offers a practical guide to setting up and using KubeMQ, with real-world application scenarios. Read if you are interested in microservices management.
🌟Best Practices🚀
Teaching machines to code: This article emphasizes the importance of "tiered documentation" for both developers and LLMs. Read for insights into the emerging need for dual-level documentation in software development, catering to both human developers and AI-driven coding assistants for optimal productivity and code quality.
Database Security - Best Practices and What You Need to Know: This article emphasizes implementing best practices like regular audits, data encryption, strict access control, and robust backup and recovery plans. Read to learn about comprehensive strategies and tools for database security.
Infrastructure as Code - What You Need to Know: IaC as a method brings consistency, reproducibility, and scalability to system management, reducing human error. Read to understand IaC's role in system deployment and management, choose between mutable and immutable methods, and explore various IaC tools for diverse infrastructure needs.
How to Make Your Awesome GitHub Profile: This comprehensive guide covers the steps to set up a profile, using GitHub Flavored Markdown and HTML for customization, and enhancing the profile with badges, icons, emojis, stats, and quotes. Read to make your profile more appealing and informative.
Error Handling in Programming: This article provides an overview of error handling in programming, detailing strategies and best practices to manage and resolve unexpected issues effectively in software development. Read for insights into effective error handling strategies.
🧠 Expert Insight 📚
Here’s an exclusive excerpt from “Chapter 16, Property-Based Testing” in the book, Soar with Haskell by Tom Schrijvers.
Test properties – a case study
In this section, we’ll explore writing different test properties by employing a small case study.
System under test
Our system under test is a compiler from a small expression language to a corresponding stack language.
We can use the following type of expression:
data Expr
= Lit Int
| Add Expr Expr
| Sub Expr Expr
| Mul Expr Expr
deriving Show
It features literals, addition, subtraction, and multiplication.
The stack language features similar functionality, but the parameters of the binary operators are not explicitly given. Instead, they are taken from a stack:
data Instr = Push Int | Plus | Minus | Times
type Prog = [Instr]
type Stack = [Int]
A program in the stack language is a sequence of instructions that are executed consecutively:
exec :: Prog -> Stack -> Maybe Stack
exec [] s = Just s
exec (i:is) s = instr i s >>= exec is
Each instruction transforms the stack by taking whatever operands it needs from the top of the stack and pushing its result back on the stack:
instr :: Instr -> Stack -> Maybe Stack
instr (Push n) stack = Just (n : stack)
instr Plus (x : y : stack) = Just (x + y : stack)
instr Minus (x : y : stack) = Just (x - y : stack)
instr Times (x : y : stack) = Just (x * y : stack)
instr _ _ = Nothing
We use the Maybe monad to model the fact that execution may crash when there are not enough operands available on the stack.
The compiler between the two languages is defined as follows:
compile :: Expr -> Prog
compile (Lit n) = [Push n]
compile (Add e1 e2) = compile e1 ++ compile e2 ++ [Plus]
compile (Sub e1 e2) = compile e1 ++ compile e2 ++ [Minus]
compile (Mul e1 e2) = compile e1 ++ compile e2 ++ [Times]
It maps a literal to a program that pushes the literal’s value on the stack. For the binary operators, the corresponding stack program first sets up the two operands on the stack and then performs the operation that fetches them from the stack.
Properties
There is a subtle bug hidden in the preceding code. Our mission now is to uncover it by exploring various properties.
Same size
The first property we propose is based on the observation that the stack program must perform the same number of operations as the original expression. The former can be obtained as the program’s length. For the latter, we must define a new function:
exprSize :: Expr -> Int
exprSize (Lit _) = 1
exprSize (Add e1 e2) = 1 + exprSize e1 + exprSize e2
exprSize (Sub e1 e2) = 1 + exprSize e1 + exprSize e2
exprSize (Mul e1 e2) = 1 + exprSize e1 + exprSize e2
Now, the property can be expressed as follows:
prop_same_size :: Expr -> Bool
prop_same_size e =
exprSize e == length (compile e)
Before we can test this property, we have to provide a generator for expressions:
instance Arbitrary Expr where
arbitrary = sized go where
go :: Int -> Gen Expr
go 0 = Lit <$> arbitrary
go n = oneof
[ Lit <$> arbitrary
, Add <$> go (n `div` 2) <*> go (n `div` 2)
, Sub <$> go (n `div` 2) <*> go (n `div` 2)
, Mul <$> go (n `div` 2) <*> go (n `div` 2) ]
This generator follows a similar strategy as that for binary trees. We use the size parameter to restrict the size of the expression rather than the height, as we did for the trees. When the size is zero, we only generate a literal. When the size is non-zero, we generate each constructor with the same likelihood. In the case of the binary operators, we halve the size for each parameter.
With this generator in place, we can see that the size property succeeds on 100 tests:
*Main> quickCheck prop_same_size
+++ OK, passed 100 tests.
We have to look further.
Successful execution
While a stack program may fail because the stack does not contain enough values, this should never be the case for compiled expressions. Indeed, compiled expressions should be self-contained and put all the values they need on the stack and in time for when they are needed. We can express this with the following property:
prop_exec_compile_succeeds :: Expr -> Bool
prop_exec_compile_succeeds e =
isJust (exec (compile e) [])
Here, we use the isJust :: Maybe a -> Bool function from the Data.Maybe library to check for success.
As it turns out, QuickCheck confirms that this property is satisfied on 100 test inputs:
*Main> quickCheck prop_exec_compile_succeeds
+++ OK, passed 100 tests.
But we can be more precise about the stack discipline of compiled expressions than this.
Stack discipline
Not only does a compiled expression push all the data it needs on the stack, but it does not push any more than it needs. At the end, we get back the original stack, with one additional value top, namely the result of the expression:
prop_stack_discipline :: Expr -> Stack -> Bool
prop_stack_discipline e s =
fmap tail (exec (compile e) s) == Just s
Here, we executed the compiled expression with an arbitrary initial stack. At the end, we check that the tail of the final stack is the same as the initial stack.
This property holds too:
*Main> quickCheck exec_stack_discipline
+++ OK, passed 100 tests.
Yet, we have not said anything about the element at the top of the final stack.
Same result
Of course, we expect that the stack program yields the same result as the original expression. Hence, we can compare against the basic evaluation function for expressions:
eval :: Expr -> Int
eval (Lit n) = n
eval (Add e1 e2) = eval e1 + eval e2
eval (Sub e1 e2) = eval e1 - eval e2
eval (Mul e1 e2) = eval e1 * eval e2
We state that the result should be the same with the following property:
prop_same_result :: Expr -> Bool
prop_same_result e =
exec (compile e) [] == Just [eval e]
Now, we are getting somewhere:
*Main> quickCheck prop_same_result
*** Failed! Falsified (after 6 tests):
Add (Sub (Lit 3) (Sub (Lit 5) (Lit (-2)))) (Add (Add (Lit (-2)) (Lit (-4))) (Add (Lit 3) (Lit 3)))
We have found a counterexample! Unfortunately, it is rather large.
Shrinking
Let’s simplify the counterexample by implementing a shrinking approach:
shrink (Lit n) = [Lit n' | n' <- shrink n]
shrink (Add e1 e2) =
[e1, e2] ++ [Add e1' e2' | (e1', e2') <- shrink (e1, e2)]
shrink (Sub e1 e2) =
[e1, e2] ++ [Sub e1' e2' | (e1', e2') <- shrink (e1, e2)]
shrink (Mul e1 e2) =
[e1, e2] ++ [Mul e1' e2' | (e1', e2') <- shrink (e1, e2)]
This is essentially the same shrinking approach we used for binary trees. This formulation is a little bit more compact as it uses the shrinking strategy for tuples to combine the two alternatives of shrinking either the left or the right subexpression.
This shrinking pays off as we get a much smaller counterexample:
*Main> quickCheck prop_same_result
*** Failed! Falsified (after 4 tests and 5 shrinks):
Sub (Lit 0) (Lit 1)
The test case says that 0 - 1 goes wrong. We can confirm this ourselves:
*Main> eval (Sub (Lit 0) (Lit 1))
-1
*Main> exec (compile (Sub (Lit 0) (Lit 1))) []
Just [1]
We can see that the evaluator behaves correctly, but the compiler performs the subtraction the wrong way around. The problem is in this line of code:
compile (Sub e1 e2) = compile e1 ++ compile e2 ++ [Minus]
The two subexpressions should appear in the opposite order:
compile (Sub e1 e2) = compile e2 ++ compile e1 ++ [Minus]
Indeed, by coming second, the value for e1 will appear on top of that for e2. This is consistent with the corresponding instr case:
instr Minus (x : y : stack) = Just (x - y : stack)
The order does not matter for addition and multiplication, but morally, we should alter their compile cases as well.
Either way, with the problem fixed, our property now succeeds:
*Main> quickCheck prop_same_result
+++ OK, passed 100 tests.
In fact, for good measure, we should rerun all properties. This is facilitated by adding the following two lines at the end of the program:
return []
runTests = $quickCheckAll
Here, quickCheckAll is a TemplateHaskell macro that gathers all the properties in the module, provided their names start with prop_. To enable TemplateHaskell, you also need to include the following pragma at the beginning of the file:
{-# LANGUAGE TemplateHaskell #-}
The line with return [] is a TemplateHaskell peculiarity that ensures the subsequent line can see all the prior definitions.
With runTests in place, we can run all tests in one go:
*Main> runTests
=== prop_same_size from Compiler.hs:44 ===
+++ OK, passed 100 tests.
=== prop_exec_compile_succeeds from Compiler.hs:48 ===
+++ OK, passed 100 tests.
=== prop_stack_discipline from Compiler.hs:52 ===
+++ OK, passed 100 tests.
=== prop_same_result from Compiler.hs:56 ===
+++ OK, passed 100 tests.
True
Isn’t this satisfactory?
Packt subscribers can continue reading for free here. Soar with Haskell by Tom Schrijvers, was published in December 2023. You can buy the book here.
🛠️ Useful Tools ⚒️
storybook: a tool for frontend developers to build UI components and pages in isolation, facilitating UI development, testing, and documentation, and is widely used across numerous teams and frameworks.
DataProfiler: a Python library for efficient data analysis and sensitive data detection, offering automatic data loading, comprehensive profiling, and entity recognition.
PurpleLlama: a project combining tools and evaluations for responsible use of open generative AI models, focusing on cybersecurity and safeguarding inputs/outputs.
vanna: an open-source Python framework for SQL generation using Retrieval-Augmented Generation (RAG), facilitating user-friendly interaction with databases via natural language queries.
That’s all for today.
We have an entire range of newsletters with focused content for tech pros. Subscribe to the ones you find the most useful here. Complete ProgrammingPro archives can be found here. Complete PythonPro archives are here.
📢 If your company is interested in reaching an audience of developers, software engineers, and tech decision makers, you may want to advertise with us.
If you have any feedback, leave a comment below.