ProgrammingPro #32: GitHub AI, Rust Revolution, Lua with C++, and Git Demystified
Bite-sized actionable content, practical tutorials, and resources for programmers
“Empty your mind, be formless. Shapeless, like water. If you put water into a cup, it becomes the cup. You put water into a bottle and it becomes the bottle. You put it in a teapot, it becomes the teapot. Now, water can flow or it can crash. Be water, my friend.”
– Bruce Lee (1971), Bruce Lee's Lost Interview in the Pierre Berton Show
Did you know that Bruce Lee had some killer insights about programming? Welcome to today’s issue of ProgrammingPro!
In today’s issue we're diving into insights ranging from GitHub's AI revolution to Microsoft's Rust rendezvous, critically analyzing the term Bad Programmers, optimizing C++ and pairing up with AI. But that is not all and here are my top 5 picks:
Cruise recalls all 950 of its self driving cars 🚗 to fix their programming
🔧Leaders are Tool Builders: Why I Wrote My Own Javascript UI Framework
Build a Lightweight 🌬️Code Generator with TypeScript and JSON Imports
Ever wondered how to extend your C++ programs at runtime without recompiling the entire codebase? Integrate Lua to C++ is the go-to guide for achieving this seamlessly with Lua code. And just for our readers, we have a exclusive excerpt from the book on multithreading in Lua and C++ in our Expert Insight section. But does Lua support multithreading🤔? Scroll down to find out.
Stay awesome!
Divya Anne Selvaraj
Editor-in-Chief
PS: If you want us to find a tutorial for you the next time or give us any feedback, jl! If you are looking for Python stuff, go here.
🗞️News, 💡Opinions, and 🔎Analysis
🗞️News
GitHub Universe 2023 opening keynote: In this high-velocity 🚀 keynote, GitHub's CEO Thomas Dohmke talks about redefining the platform in the age of AI, particularly showcasing groundbreaking advancements in GitHub Copilot. Watch for insights into Copilot's integration throughout the workflow, its application in GitHub.com, and significant partnerships, and peek 👀 into the future of coding.
Microsoft just gave up C/C++ (use 🦀 Rust!): Microsoft's aggressive embrace of Rust continues with a $10 million investment in developer tools, aiming to combat memory bugs and enhance security. Watch to understand how this move solidifies Rust's importance within Microsoft and creates opportunities for developers.
Firefox 🦊 Development Is Moving From Mercurial To Git: While the primary repository will move to Git hosted on GitHub, the contribution workflow will remain unchanged, utilizing tools like Bugzilla and moz-phab. Read to learn how the transition will occur and why the move is being made.
The Escalating Cyber Front in the ISRAEL-ISISHAMAS Conflict: This article discusses the emergence of the BiBi-Linux Wiper malware and its destructive capabilities, as well as the role of hacktivist groups like "Haghjhoyan" and "Soldiers of Solomon" in using social engineering and ransomware. Read to stay informed and vigilant in the face of this evolving threat landscape, and take preventive steps.
Q3 2023 Chrome Security Updates: This highlights Chrome Security's initiatives, covering enhancements to web security, TLS, automation, and improvements in the web PKI's Certificate Transparency infrastructure. Read to ensure your applications remain secure and up-to-date with the latest technologies and practices.
Cruise recalls all 950 of its self driving cars to fix their programming: GM's self-driving car subsidiary is recalling its fleet to address a software issue related to how the cars respond to impacts. Read to learn more about the incident which triggered the recall and understand why software improvements are crucial in the real world.
💡Opinions and Analysis🔎
I'm a hypochondriac, but It's About Programming: This article explores the concept of the so called "Bad Programmers" in the software development industry and the scrutiny they face. Read to learn about the prevalence of strong opinions and the line between legitimate criticism and mean-spiritedness in the profession.
Beyond YAML: Why Semantic Layers Need Real Programming Languages: This article discusses the limitations of using YAML for semantic layers and advocates for the use of full-fledged programming languages to enhance developer experience and data modeling efficiency. Read to understand the importance of fast feedback loops and error detection in creating a more productive environment.
Leaders are Tool Builders: Why I Wrote My Own Javascript UI Framework: In this article, Kenn Costales emphasizes the importance of technical leaders building their own tools and systems tailored to their company's specific needs, highlighting that off-the-shelf solutions may not align with unique challenges and objectives. Read for insights into overcoming challenges and fostering innovation.
How to Do a TypeScript Conversion: This article dives into the common question of TypeScript conversion—convert-as-you-go or follow the dependency graph? Read to learn why the author who draws from the experience of converting a massive app for LinkedIn, warns against the seemingly easier route.
🎓 Tutorials and Guides🤓
Are you interested in optimizing C++ code by generating arrays at compile-time with lambdas?: This short article explores how to create a 256-byte Boolean array for efficient character set checks. Read to compare two methods using constexpr and see how they stack up against the default approach.
Unlock Symbolic Computation with GiNaC in C++: This tutorial provides insight into using GiNaC, an open framework for symbolic computation, for generating and manipulating symbolic expressions. Read if you are a developer or engineer working on complex calculations or numerical applications.
Writing A Jekyll Converter for Literate Programming: This article discusses the development of a Jekyll converter, highlighting the implementation details and how it facilitates embedding code and explanations in blog posts. Read if you are interested in combining code and documentation in a seamless manner for more effective and informative technical content on Jekyll-based blogs.
Mastering DOM manipulation with vanilla JavaScript: From basic tasks like changing favicons dynamically to advanced techniques such as creating custom cursors and smooth scrolling, this resource covers it all. Read to enhance your ability to navigate and manipulate the Document Object Model effectively.
Rust for JavaScript Developers: An Overview of Testing: This article contrasts testing setups in JavaScript, using frameworks like Mocha ☕ and Jest 🃏, with Rust's built-in testing capabilities through Cargo. Read if you are navigating testing strategies in both languages and want insights into unit and integration testing.
Ever faced sneaky JavaScript errors in your Playwright tests?: This short video tutorial transforms a test case into a reusable JavaScript error tracking powerhouse, ensuring your tests notice when your frontend is misbehaving. Watch to say goodbye to overlooking critical issues and adopt the efficiency of Playwright fixtures.
Build a Lightweight Code Generator with TypeScript and JSON Imports: This tutorial will guide you in constructing a lightweight code generator with TypeScript for seamless adaptation to dynamic changes in external systems, using Postmark email templates as a case study. Read if you need to deal with evolving third-party services and find a practical approach to maintain accuracy and efficiency.
🔑 Secret Knowledge: Learning Resources🔬
Faster than Rust and C++: the PERFECT hash table: This video covers topics like how hash tables function, the importance of optimizing hash functions for better performance, and techniques for avoiding collisions. Watch to explore various libraries and approaches for perfect hash tables, and more.
Lifetime Safety in C++: Past, Present and Future: This video discusses current mitigations for enhancing code safety, such as warnings, static analysis checks from MSVC, Clang, and GCC, dynamic analysis tools, and potential changes to the C++ language. Watch to to discover practical ways to enhance the safety of your C++ code without sacrificing performance and flexibility.
Composition on Tiny Embedded Systems in C++: This live demonstration shows how to create a remote-controlled RGB lighting controller, covering aspects like low-level hardware access, interrupt handling, logging, initialization, and message processing. Watch for C++ and design concepts, including compile-time constructs for efficient abstractions, component decoupling strategies, and more.
A new way to bring garbage collected programming languages efficiently to WebAssembly: This article explores the concept of efficiently bringing garbage collected programming languages to WebAssembly using the WasmGC approach. Read to understand how this approach works and learn about advantages, including better memory management, cycle collection, and optimizations.
Implementing Protected Members in JavaScript Classes: While JavaScript lacks native support for protected members, the article recommends a workaround to ensure encapsulation, clarity, and flexibility. Read for a code walkthrough that details superclass and subclass implementation, along with challenges and benefits.
The 4 Best Scrapy Libraries to Scrape JS Heavy Websites: This article highlights four key Scrapy libraries for Javascript rendering: Scrapy Playwright, Scrapy Splash, Scrapy Selenium, and Scrapy Puppeteer. Read if you are dealing with modern web applications to navigate the complexities of data extraction from single page applications built with React.js, Angular.js, Vue.js, and more.
Go 🐭mouseless🐭: 10x Developer Workflow on Windows: This article delves into a workflow, emphasizing efficiency through keyboard shortcuts and a meticulously configured system. Read to ditch the mouse for shortcuts and explore powerful tools like Neovim and tiling window managers.
Ever find yourself tangled in the labyrinth of Git jargon?: This guide will help you Unravel the mysteries behind "detached HEAD state," decipher the dance of "ours" and "theirs" in merges, demystify the subtle nuances of HEAD^, HEAD~, and more. Read to untangle the web 🕸️ of Git terminology.
🧠 Expert Insight 📚
Here’s an excerpt from Chapter 11, Multithreading with Lua in the book Integrate Lua with C++ by Wenhuan Li.
Multithreading in C++
What is multithreading?
There are a few definitions, depending on the point of view. From the CPU’s perspective, a multi-core processor that can execute multiple threads of instructions concurrently is real multithreading. From an application’s perspective, using multiple threads is multithreading.
From a developer’s perspective, more focus might be on thread safety and various synchronization mechanisms, which are not multithreading itself, but its implications.
In this section, we will learn how to use Lua with C++’s native multithreading support. Each C++ thread will have its own Lua state. Because the Lua library does not keep any state and Lua states are not shared among different threads, this is thread-safe.
How does C++ support multithreading?
Since C++11, the standard library supports multithreading with std::thread. Each std::thread instance represents a thread of execution. The most important thing to provide to a thread is the thread function. This is what a thread executes. In its simplest form, we can create a thread as follows:
void threadFunc(...) {}
std::thread t(threadFunc, ...);
Here, we passed a C++ function as the thread function to create a thread. The function can optionally take arguments and the std::thread constructor will forward the arguments to the thread function. After the thread is created, the thread function starts to execute in its own thread. When the thread function finishes, the thread is ended.
We can also use a class member function or a class static member function as the thread function by invoking different constructors. You can refer to a C++ reference manual to learn more about std::thread.
Before C++11
In the era before C++11, there was no standard multithreading support. People had to use third-party libraries or implement their own with a low-level library, such as pthreads.
This type of multithreading is unlikely to surprise you. This is the type of multithreading that people have talked about and have used most, which is preemptive multithreading. The thread function can be paused at any time and resumed at any time.
Next, we will explore a real example to see C++ multithreading in action.
Using multiple Lua instances
In this section, we will implement a thread function in which we’ll execute a Lua script. Then, we will create multiple threads to execute this same thread function.
Based on the source code from Chapter 10, wipe main.cpp clean and add the following code:
#include "LuaExecutor.h"
#include "LoggingLuaExecutorListener.h"
#include <iostream>
#include <mutex>
#include <thread>
#include <vector>
auto listener = std::make_unique
<LoggingLuaExecutorListener>();
std::mutex coutMutex;
Here, we added the necessary headers. listener is the Lua executor listener and will be shared for all Lua executor instances. coutMutex is a mutex for printing results with std::cout, whose usage we will see next.
Next, implement the thread function, as follows:
void threadFunc(int threadNo, int a, int b, int c)
{
auto lua = std::make_unique<LuaExecutor>(*listener);
lua->execute("function add_params(a, b, c) return a + b
+ c end");
auto result = lua->call("add_params",
LuaNumber::make(a), LuaNumber::make(b),
LuaNumber::make(c));
std::lock_guard<std::mutex> lock(coutMutex);
std::cout << "[Thread " << threadNo << "] "
<< a << "+" << b << "+" << c << "="
<< std::get<LuaNumber>(result).value
<< std::endl;
}
This creates a list of threads and waits for the threads to finish execution.
In the first for loop, we use std::vector::emplace_back to create the threads at the end of the vector in place. Internally, for most C++ implementations, it uses placement new and invokes std::thread(threadFunc, i, a, a + 1, a + 2). We do this because std::thread is not copy-constructible. Understandably, it does not make sense to copy a thread.
In the second for loop, we use std::thread::join to wait for all threads to finish execution. The main function runs in the main thread of the application process. When main exits, all other threads will be aborted, even if they have not finished execution.
Next, we’ll test our example.
Testing it out
Compile and execute the project. You should see an output similar to the following:
Chapter11 % ./executable
[Thread 2] 4+5+6=15
[Thread 3] 7+8+9=24
[Thread 5] 13+14+15=42
[Thread 1] 1+2+3=6
[Thread 4] 10+11+12=33
If you run the project multiple times, you will see the order of the results from different threads changes. This verifies that we are using Lua with multiple threads.
For most projects, when integrating Lua into C++, this mechanism should suffice for multithreading. This is multithreading with C++. The Lua part just works without any additional effort. Each C++ thread has its own Lua instance and executes its copy of the Lua scripts. Different Lua instances do not interfere with or know about each other.
Next, we will explore multithreading in Lua.
Multithreading in Lua
To understand multithreading in Lua, let’s begin with a fundamental question.
How does Lua support multithreading?
Lua does not support multithreading. Period.
But we cannot finish this section yet. We will explain this further with two approaches – a contemporary one and an old-school one.
The contemporary approach
Lua is a scripting language and it does not support preemptive multithreading. It simply does not provide a library function to create a new thread, so there is no way to do it.
Nowadays, CPUs and operating systems are designed around preemptive multithreading – that is, a thread of execution can be paused and resumed at any time. A thread has no control over its execution schedule.
However, Lua provides a mechanism for cooperative multithreading with coroutines. In a cooperative multithreading environment, the thread of execution is never preempted. Only when the thread willingly gives up its execution can another thread start to execute using the same CPU core. An application component that can be executed this way is called a coroutine, which is usually a function.
coroutine is also very popular with Kotlin for Android and backend development.
Cooperative multithreading
When we talk about threads, most of the time, the implication is that they are threads for CPU cores to execute. When we talk about cooperative multithreading, in some cases, such as the one for Lua, you may find that there is only one thread being executed and one CPU core used, even with coroutines. Arguably, this is not multithreading at all. But we do not need to judge. We need to understand this because multiple terms can be used for this in different contexts. We can also call this cooperative multitasking, which is technically more accurate from a historical point of view.
Let’s see Lua’s coroutine in action and explain it more.
Implementing a Lua coroutine
Replace the content of script.lua with the following code:
function create_square_seq_coroutine(n)
return coroutine.create(function ()
for i = 1, n do
coroutine.yield(i * i)
end
end)
end
create_square_seq_coroutine creates a coroutine with coroutine.create, which, in turn, takes an anonymous function as its argument. You can roughly think that the inner anonymous function is coroutine. The inner function runs a loop and yield the squares from 1 to n.
You can only use yield with coroutines. A coroutine will stop execution when it reaches a yield statement. The values provided to yield will be returned to the call site, similar to what return does. The next time you execute coroutine, it will resume the execution from where it yielded until it reaches another yield statement or a return statement.
Let’s start an interactive Lua interpreter to test our coroutine:
Chapter11 % ../lua/src/lua
Lua 5.4.6 Copyright (C) 1994-2023 Lua.org, PUC-Rio
> dofile("script.lua")
> co = create_square_seq_coroutine(3)
> coroutine.resume(co)
true 1
> coroutine.resume(co)
true 4
> coroutine.resume(co)
true 9
> coroutine.resume(co)
true
> coroutine.resume(co)
false cannot resume dead coroutine
Here, we create a coroutine to return the squares from 1 to 3. The first time we resume coroutine, it starts to execute from the beginning and returns two values, true and 1. true is from coroutine.resume and means that coroutine is executed without any error. 1 is what coroutine yielded. The next time we resume coroutine, the loop continues with the next iteration and returns 4. Pay special attention to the line when coroutine.resume only returns one value. The loop has finished but there is still code to be executed for coroutine, such as the implicit return statement. So, coroutine.resume returns true. After that, coroutine has finished and cannot be resumed and coroutine.resume will return false with an error message.
If this is the first time you have used coroutine with any programming language, this may seem magical and non-logical to you. How could a function, not in a thread, not reach its end and get executed from the middle of it again? I will explain why this is so ordinary (but do say you know coroutine and why it is so glorious in an interview) in the last part of this section. Before that, let’s explore another example to see a case in which coroutine can be very useful.
Lua coroutine as iterator
We have seen how to use iterators with the generic for to simplify our lives, for example, ipairs.
But what is an iterator?
An iterator is something that can be called again and again to produce values until there are no more to produce. For Lua, iterator returns an iterator function that can be called again and again until it returns nil or nothing.
Based on coroutine that we have just implemented to generate a sequence of squares, let’s build an iterator. In script.lua, add another function, as follows:
function square_seq(n)
local co = create_square_seq_coroutine(n)
return function()
local code, value = coroutine.resume(co)
return value
end
end
square_seq is a Lua iterator as it returns its inner function as an iterator function. The inner function continues to resume the coroutine created with create_square_seq_coroutine. It is the caller’s responsibility to stop calling the iterator function when the iterator function has returned nil or nothing.
Let’s test this iterator in an interactive Lua interpreter:
Chapter11 % ../lua/src/lua
Lua 5.4.6 Copyright (C) 1994-2023 Lua.org, PUC-Rio
> dofile("script.lua")
> for v in square_seq(3) do print(v) end
1
4
9
You can see that three values are printed as expected for 1, 2, and 3.
And by looking at the usage, you cannot even tell if any coroutine or cooperative multithreading is involved. This, I think, is one of the examples where this programming paradigm can be more valuable than preemptive multithreading.
So far, we have explored Lua coroutine and Lua iterator. They can be more complex, but these examples are enough to show you how they work. You can refer to the Lua reference manual to learn more about coroutine and iterator.
Integrate Lua with C++ by Wenhuan Li was published in October 2023. You can get the book here.
🛠️ Useful Tools ⚒️
aider: a command line tool facilitating AI-powered pair programming, utilizing models like GPT-3.5 and GPT-4 for code editing in local git repositories across different programming languages, allowing for code changes and more.
guava: a core Java library from Google, which provides various collection types, immutable collections, a graph library, and utilities for concurrency, I/O, hashing, primitives, strings, and more, supporting both JRE and Android environments.
plotly.js: a JavaScript data visualization library that enables the creation of various charts and visualizations, widely used in Python and R ecosystems (Plotly.py and Plotly.R), offering consulting services for integration, dashboard development, and feature enhancements.
axflow: a TypeScript framework that facilitates modular and scalable development of AI applications, emphasizing a code-first approach for building robust natural language-powered features.
fury: a high-performance serialization framework, delivers up to 170x speed improvement with JIT compilation and zero-copy, offering a seamless replacement for Java serialization frameworks and ensuring compatibility.
📢 If your company is interested in reaching an audience of developers, software engineers, and tech decision makers, you may want to advertise with us.
We have an entire range of newsletters with focused content for tech pros. Subscribe to the ones you find the most useful here. Complete ProgrammingPro archives can be found here. Complete PythonPro archives are here.
If you have any feedback, leave your comments below!