Embedding Scripting Languages
C++ scripting integration — embedding Lua (sol2), embedding Python (pybind11), JavaScript (QuickJS), Lua coroutines, hot reloading, sandboxing untrusted scripts, and exposing C++ APIs.
TL;DR
Embedding a scripting language lets users extend your app without recompiling. Lua is the standard for game/embedded use (small, fast, designed to be embedded). Python (via pybind11) is better for scientific/data tooling. Both allow hot-reload of scripts at runtime.
Lua via sol2 (Recommended)
sol2 is a modern C++ wrapper around the Lua C API:
#include <sol/sol.hpp>
int main() {
sol::state lua;
lua.open_libraries(sol::lib::base, sol::lib::math, sol::lib::string);
// Execute a script string
lua.script(R"(
function greet(name)
return "Hello, " .. name .. "!"
end
)");
// Call Lua function from C++
std::string result = lua["greet"]("World");
std::println("{}", result); // Hello, World!
// Set/get global variables
lua["player_speed"] = 5.0;
double speed = lua["player_speed"];
// Run a script file
lua.script_file("game_logic.lua");
}Exposing C++ Types to Lua (sol2)
struct Vec2 { float x, y; };
struct Player {
Vec2 pos{};
std::string name;
int health = 100;
void move(float dx, float dy) { pos.x += dx; pos.y += dy; }
bool is_alive() const { return health > 0; }
};
sol::state lua;
lua.open_libraries(sol::lib::base);
// Register Vec2
lua.new_usertype<Vec2>("Vec2",
sol::constructors<Vec2(), Vec2(float, float)>(),
"x", &Vec2::x,
"y", &Vec2::y
);
// Register Player
lua.new_usertype<Player>("Player",
"name", &Player::name,
"health", &Player::health,
"pos", &Player::pos,
"move", &Player::move,
"is_alive", &Player::is_alive
);
Player p;
p.name = "Hero";
lua["player"] = &p; // expose existing C++ object to Lua
lua.script(R"(
player:move(1.0, 2.0)
print(player.name .. " at " .. player.pos.x .. "," .. player.pos.y)
)");Calling C++ from Lua (Callbacks)
// Register a C++ function in Lua
lua.set_function("spawn_enemy", [](std::string type, int level) -> int {
std::println("spawning {} at level {}", type, level);
return 42; // entity ID
});
lua.script(R"(
local id = spawn_enemy("goblin", 3)
print("spawned entity " .. id)
)");
// Register a table of functions (namespace-like)
lua["Engine"] = lua.create_table_with(
"log", [](std::string msg) { std::println("[LUA] {}", msg); },
"time", []() -> double { return /* get time */0.0; }
);
lua.script("Engine.log('hello from Lua')");Hot Reloading Scripts
class ScriptEngine {
sol::state lua_;
std::filesystem::file_time_type last_modified_;
std::filesystem::path script_path_;
public:
void load(std::filesystem::path path) {
script_path_ = path;
reload();
}
void reload() {
try {
lua_.script_file(script_path_.string());
last_modified_ = std::filesystem::last_write_time(script_path_);
std::println("reloaded {}", script_path_.string());
} catch (const sol::error& e) {
std::println(stderr, "script error: {}", e.what());
}
}
void tick() {
// Check if script changed
auto mtime = std::filesystem::last_write_time(script_path_);
if (mtime != last_modified_) reload();
// Call the Lua update function
if (auto fn = lua_["update"]; fn.valid())
fn(/* delta_time */ 0.016);
}
};Python via pybind11
For scientific/ML tooling where Python ecosystem is needed:
#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
namespace py = pybind11;
// Expose a C++ function to Python
PYBIND11_MODULE(mymodule, m) {
m.def("add", [](int a, int b) { return a + b; }, "Add two integers");
py::class_<Player>(m, "Player")
.def(py::init<std::string>())
.def_readwrite("health", &Player::health)
.def("move", &Player::move)
.def("is_alive", &Player::is_alive);
}
// Then in Python:
// import mymodule
// p = mymodule.Player("Hero")
// p.move(1.0, 0.0)Lua Coroutines from C++
Lua coroutines are cooperative threads that can yield back to C++. This is the standard pattern for game AI, cutscenes, and async sequences:
// Lua script: coroutine that runs over multiple frames
sol::state lua;
lua.open_libraries(sol::lib::base, sol::lib::coroutine);
lua.script(R"(
function ai_sequence()
print("start patrol")
coroutine.yield() -- pause, return to C++
print("check room")
coroutine.yield() -- pause again
print("return to base")
end
)");
// Create a coroutine from the Lua function
sol::coroutine ai = lua["ai_sequence"];
// Drive it from the C++ frame loop
void update_ai() {
if (ai.runnable()) {
auto result = ai(); // resume until next yield
if (!result.valid()) {
sol::error e = result;
std::println(stderr, "ai error: {}", e.what());
}
}
}Each call to ai() resumes the Lua function until the next coroutine.yield() — perfect for multi-step sequences driven one step per frame.
Sandboxing Untrusted Scripts
When scripts come from users or plugins, restrict what they can access:
sol::state lua;
// Open only safe libraries — NOT io, os, package, debug
lua.open_libraries(
sol::lib::base, // print, ipairs, pairs, type, tostring
sol::lib::math, // math.*
sol::lib::string, // string.*
sol::lib::table // table.*
);
// Remove dangerous functions
lua["dofile"] = sol::nil;
lua["loadfile"] = sol::nil;
lua["require"] = sol::nil;
lua["collectgarbage"] = sol::nil;
// Expose only the specific C++ API you want to allow
lua.set_function("get_score", [&]() { return game_.score(); });
lua.set_function("spawn_particle", [&](float x, float y) {
effects_.spawn(x, y);
});
// Set CPU/memory limits (Lua 5.4+ debug hook)
lua_sethook(lua.lua_state(), [](lua_State*, lua_Debug*) {
// Called every N instructions — check budget
static int count = 0;
if (++count > 100000)
luaL_error(lua.lua_state(), "script exceeded instruction budget");
}, LUA_MASKCOUNT, 1000);Embedding JavaScript — QuickJS
QuickJS is a compact, embeddable JavaScript engine (ES2023 support) with a C API. Useful when Lua's syntax is unfamiliar to your target audience.
#include "quickjs.h"
// Initialize runtime (one per process) and context (one per "tab")
JSRuntime* rt = JS_NewRuntime();
JSContext* ctx = JS_NewContext(rt);
// Expose a C++ function to JS
static JSValue js_add(JSContext* ctx, JSValue, int argc, JSValue* argv) {
int a = 0, b = 0;
JS_ToInt32(ctx, &a, argv[0]);
JS_ToInt32(ctx, &b, argv[1]);
return JS_NewInt32(ctx, a + b);
}
JSValue global = JS_GetGlobalObject(ctx);
JS_SetPropertyStr(ctx, global, "add",
JS_NewCFunction(ctx, js_add, "add", 2));
JS_FreeValue(ctx, global);
// Run JavaScript
const char* script = "add(3, 4)";
JSValue result = JS_Eval(ctx, script, strlen(script), "<inline>", JS_EVAL_TYPE_GLOBAL);
int val = 0;
JS_ToInt32(ctx, &val, result);
std::println("JS result: {}", val); // 7
JS_FreeValue(ctx, result);
JS_FreeContext(ctx);
JS_FreeRuntime(rt);QuickJS has no external dependencies, compiles to ~210KB, and supports async/await — useful for Node.js-familiar audiences.
CMake Setup
# Lua + sol2
find_package(Lua REQUIRED)
FetchContent_Declare(sol2 GIT_REPOSITORY https://github.com/ThePhD/sol2.git GIT_TAG v3.3.0)
FetchContent_MakeAvailable(sol2)
target_link_libraries(myapp PRIVATE ${LUA_LIBRARIES} sol2)
target_include_directories(myapp PRIVATE ${LUA_INCLUDE_DIR})
# pybind11
find_package(pybind11 CONFIG REQUIRED)
pybind11_add_module(mymodule src/bindings.cpp)
target_link_libraries(mymodule PRIVATE mylib)Lua C API Deep Dive — lua_State Lifecycle and C Closures
sol2 is the recommended wrapper, but understanding the raw Lua C API lets you write zero-overhead bindings and diagnose hard crashes. Every interaction is mediated through a stack: you push values, call functions, and pop results. A C closure captures upvalues at the C level, not the Lua level.
extern "C" {
#include <lua.h>
#include <lualib.h>
#include <lauxlib.h>
}
// --- lua_State lifecycle ---
lua_State* L = luaL_newstate(); // allocate + init the VM
luaL_openlibs(L); // open all standard libraries
// Execute a string — result and errors go on the stack
if (luaL_dostring(L, "return 1 + 2") != LUA_OK) {
std::string err = lua_tostring(L, -1);
lua_pop(L, 1);
throw std::runtime_error("Lua: " + err);
}
// TOS (top-of-stack) = 3
double result = lua_tonumber(L, -1);
lua_pop(L, 1); // always balance the stack
lua_close(L); // free all memory — only call once, after all threads are done
// --- C closure: a C function that remembers state via upvalues ---
static int counter_inc(lua_State* L) {
// upvalue 1 holds the current count
int n = (int)lua_tointeger(L, lua_upvalueindex(1));
lua_pushinteger(L, n + 1);
lua_replace(L, lua_upvalueindex(1)); // update upvalue
lua_pushinteger(L, n + 1); // return new value
return 1; // number of return values
}
// Register the closure with an initial upvalue of 0
lua_pushinteger(L, 0); // upvalue initial value
lua_pushcclosure(L, counter_inc, 1); // 1 upvalue
lua_setglobal(L, "counter");
luaL_dostring(L, "print(counter(), counter(), counter())"); // 1 2 3
// --- Protected call (lua_pcall) for safe error recovery ---
lua_getglobal(L, "might_error"); // push function
lua_pushnumber(L, 42); // push argument
int status = lua_pcall(L, 1, 1, 0); // 1 arg, 1 return, no error handler
if (status != LUA_OK) {
std::println(stderr, "Lua error: {}", lua_tostring(L, -1));
lua_pop(L, 1);
}Stack discipline rule: every lua_push* increments the top; every lua_pop decrements it. Unbalanced stacks cause silent data corruption or stack overflows, not crashes.
Python Embedding — Py_Initialize and PyObject*
Embedding CPython lets C++ code call Python functions, import packages (numpy, scipy, matplotlib), and mix Python and C++ objects in the same process. The GIL must be held for all Python API calls; release it with Py_BEGIN_ALLOW_THREADS during CPU-heavy C++ sections.
#include <Python.h>
// --- Initialize once per process ---
Py_Initialize(); // starts the interpreter
// PySys_SetPath(...) if you need custom module search paths
// --- Import a module and call a function ---
PyObject* mod = PyImport_ImportModule("json"); // import json
PyObject* func = PyObject_GetAttrString(mod, "dumps"); // json.dumps
PyObject* arg = PyUnicode_FromString("{\"key\": 1}"); // string arg
// Actually call json.loads, not json.dumps (which needs a Python dict)
PyObject* loads = PyObject_GetAttrString(mod, "loads");
PyObject* pydict = PyObject_CallOneArg(loads, arg);
// pydict is now a Python dict object
Py_DECREF(loads);
Py_DECREF(arg);
Py_DECREF(pydict);
Py_DECREF(func);
Py_DECREF(mod);
// --- Call numpy from C++ ---
PyObject* np = PyImport_ImportModule("numpy");
PyObject* arange = PyObject_GetAttrString(np, "arange");
PyObject* result = PyObject_CallFunction(arange, "i", 10); // np.arange(10)
// result is a numpy array Python object
// --- Convert Python int/float to C++ ---
PyObject* py_val = PyLong_FromLong(42);
long c_val = PyLong_AsLong(py_val);
Py_DECREF(py_val);
// --- Run a Python script file ---
FILE* fp = fopen("script.py", "r");
PyRun_SimpleFile(fp, "script.py");
fclose(fp);
// --- Shutdown ---
Py_Finalize(); // cleans up all modules and frees memory| Task | Raw CPython API | pybind11 |
|---|---|---|
| Expose C++ class to Python | PyType_Ready (verbose) | py::class_<> (1 line) |
| Call Python from C++ | PyObject_Call* family | py::module_::import |
| numpy array interop | Buffer protocol | pybind11/numpy.h |
| Error handling | Check every != nullptr | Automatic exception conversion |
V8 Embedding — Isolate, Context, and Persistent Handles
V8 is Google's JavaScript engine (used in Chrome and Node.js); embedding it gives you a full ES2023 runtime with JIT compilation. The key concepts are Isolate (one per thread), Context (one per global scope), and handle scopes to prevent memory leaks.
#include <v8.h>
#include <libplatform/libplatform.h>
// --- One-time initialization ---
std::unique_ptr<v8::Platform> platform = v8::platform::NewDefaultPlatform();
v8::V8::InitializePlatform(platform.get());
v8::V8::Initialize();
// --- Create an Isolate (one per thread) ---
v8::Isolate::CreateParams params;
params.array_buffer_allocator =
v8::ArrayBuffer::Allocator::NewDefaultAllocator();
v8::Isolate* isolate = v8::Isolate::New(params);
{
v8::Isolate::Scope isolate_scope(isolate);
v8::HandleScope handle_scope(isolate); // stack frame for GC handles
// --- Create a Context ---
v8::Local<v8::Context> context = v8::Context::New(isolate);
v8::Context::Scope context_scope(context);
// --- Expose a C++ function to JavaScript ---
auto js_sqrt = [](const v8::FunctionCallbackInfo<v8::Value>& args) {
double x = args[0]->NumberValue(args.GetIsolate()->GetCurrentContext())
.FromJust();
args.GetReturnValue().Set(std::sqrt(x));
};
v8::Local<v8::FunctionTemplate> tmpl =
v8::FunctionTemplate::New(isolate, js_sqrt);
context->Global()->Set(context,
v8::String::NewFromUtf8Literal(isolate, "cppSqrt"),
tmpl->GetFunction(context).ToLocalChecked()).Check();
// --- Run JavaScript ---
v8::Local<v8::String> source =
v8::String::NewFromUtf8Literal(isolate, "cppSqrt(2.0)");
v8::Local<v8::Script> script =
v8::Script::Compile(context, source).ToLocalChecked();
v8::Local<v8::Value> result = script->Run(context).ToLocalChecked();
double val = result->NumberValue(context).FromJust();
std::println("sqrt(2) = {:.6f}", val); // 1.414214
// --- Persistent handle: survives beyond HandleScope ---
v8::Persistent<v8::Function> stored_fn;
// stored_fn.Reset(isolate, fn_local); // keep function alive across calls
// stored_fn.Reset(); // release when done
}
isolate->Dispose();
v8::V8::Dispose();V8 is large (~20MB binary). For lighter alternatives consider QuickJS (already covered) or Duktape.
ChaiScript and AngelScript — Embedded Scripting Without External Deps
ChaiScript is a header-only C++ scripting language that uses C++ types directly; AngelScript targets game developers with a strongly-typed syntax similar to C++. Neither requires an external runtime or build system change beyond adding include paths.
// --- ChaiScript: zero external deps, header-only ---
#include <chaiscript/chaiscript.hpp>
chaiscript::ChaiScript chai;
// Expose a C++ function
chai.add(chaiscript::fun([](double x) { return x * x; }), "square");
// Expose a class
chai.add(chaiscript::user_type<Vec2>(), "Vec2");
chai.add(chaiscript::constructor<Vec2(float, float)>(), "Vec2");
chai.add(chaiscript::fun(&Vec2::x), "x");
chai.add(chaiscript::fun(&Vec2::y), "y");
// Run a script
double result = chai.eval<double>("square(3.0)"); // 9.0
// ChaiScript syntax is close to JavaScript/C++
chai.eval(R"(
var v = Vec2(1.0, 2.0);
print(v.x + v.y); // 3.0
for(var i = 0; i < 5; ++i) { print(square(i)); }
)");// --- AngelScript: strongly-typed, game-oriented ---
#include <angelscript.h>
asIScriptEngine* engine = asCreateScriptEngine();
// Register a function
engine->RegisterGlobalFunction(
"float sqrt(float)",
asFUNCTION(sqrtf), asCALL_CDECL);
// Register a type
engine->RegisterObjectType("Vec2", sizeof(Vec2), asOBJ_VALUE | asOBJ_POD);
engine->RegisterObjectProperty("Vec2", "float x", offsetof(Vec2, x));
engine->RegisterObjectProperty("Vec2", "float y", offsetof(Vec2, y));
// Compile a module
asIScriptModule* mod = engine->GetModule("game", asGM_ALWAYS_CREATE);
mod->AddScriptSection("main", R"(
void update(float dt) {
Vec2 v;
v.x = sqrt(dt);
}
)");
mod->Build();
// Execute
asIScriptFunction* fn = mod->GetFunctionByName("update");
asIScriptContext* ctx = engine->CreateContext();
ctx->Prepare(fn);
ctx->SetArgFloat(0, 0.016f);
ctx->Execute();
ctx->Release();
engine->Release();| Feature | ChaiScript | AngelScript | Lua/sol2 |
|---|---|---|---|
| Header-only | Yes | No | No |
| Type safety | Dynamic | Static | Dynamic |
| Performance | Moderate | Fast | Fast |
| Syntax | JS-like | C++-like | Lua |
| Coroutine support | No | No | Yes |
Hot Reloading Scripts — File Watching Without Restart
Hot reload lets content creators or power users edit scripts and see results immediately while the C++ host keeps running. The minimal implementation polls std::filesystem::last_write_time; production systems use inotify (Linux) or FSEvents (macOS) for zero-latency notification.
#include <filesystem>
#include <sys/inotify.h>
#include <unistd.h>
// inotify-based watcher — zero CPU, event-driven
class FileWatcher {
int ifd_;
int wd_;
std::filesystem::path path_;
std::function<void()> callback_;
public:
FileWatcher(std::filesystem::path p, std::function<void()> cb)
: path_{std::move(p)}, callback_{std::move(cb)}
{
ifd_ = inotify_init1(IN_NONBLOCK | IN_CLOEXEC);
wd_ = inotify_add_watch(ifd_, path_.c_str(),
IN_CLOSE_WRITE | IN_MOVED_TO);
}
~FileWatcher() { inotify_rm_watch(ifd_, wd_); ::close(ifd_); }
// Call from event loop when ifd_ is readable (epoll/select)
void poll() {
char buf[sizeof(inotify_event) + NAME_MAX + 1];
ssize_t n = read(ifd_, buf, sizeof(buf));
if (n > 0) callback_();
}
int fd() const { return ifd_; }
};
// Integrate with a Lua engine
sol::state lua;
lua.script_file("game_logic.lua");
FileWatcher watcher{"game_logic.lua", [&] {
std::println("script changed — reloading");
try {
// Re-execute: top-level code runs again, functions are redefined
lua.script_file("game_logic.lua");
} catch (const sol::error& e) {
std::println(stderr, "reload error: {}", e.what());
// Keep old version running — don't crash
}
}};
// In main event loop:
// epoll_add(watcher.fd(), [&]{ watcher.poll(); });State migration caveat: re-executing a script reinitializes all globals. Preserve host-managed state (player position, scores) in C++ and expose it as read-only to scripts.
Sandboxing Scripts — API Surface Limiting
Never give an untrusted script access to the file system, network, or OS process API. The strategy is deny-by-default: open no standard libraries initially, then whitelist exactly the functions the script is allowed to call.
// Lua: minimal safe environment
sol::state lua;
// Open NOTHING by default — not even base
// Then selectively add only what you need:
lua.open_libraries(sol::lib::math); // math.sin, math.floor, etc.
lua.open_libraries(sol::lib::string); // string.format, string.find
lua.open_libraries(sol::lib::table); // table.insert, table.sort
// Explicitly whitelist base functions (print is OK, load/dofile are not)
lua.set_function("print", [](sol::variadic_args args) {
for (auto a : args) std::print("{}\t", (std::string)a);
std::println("");
});
lua.set_function("tostring", [&](sol::object o) -> std::string {
return lua["tostring"](o);
});
lua.set_function("tonumber", [](sol::object o) -> sol::object {
// wrap tonumber safely
return o;
});
// Remove dangerous globals even if they somehow appeared
for (const char* fn : {"load","loadstring","dofile","loadfile",
"require","rawget","rawset","debug","io","os"}) {
lua[fn] = sol::nil;
}
// CPU time limit via debug hook
static std::atomic<bool> timed_out{false};
lua_sethook(lua.lua_state(),
[](lua_State* L, lua_Debug*) {
if (timed_out.load())
luaL_error(L, "execution time limit exceeded");
},
LUA_MASKCOUNT, 10'000); // called every 10k instructions
auto timer = std::async(std::launch::async, [] {
std::this_thread::sleep_for(std::chrono::milliseconds(100));
timed_out.store(true);
});
try {
lua.safe_script(user_provided_code);
} catch (const sol::error& e) {
std::println(stderr, "script rejected: {}", e.what());
}
timed_out.store(false);
// AngelScript: register only whitelisted modules
// engine->RegisterGlobalFunction("float sqrt(float)", ...);
// Do NOT register file I/O, network, or reflection functionsDebugging Embedded Scripts — Source Maps and Remote Debugging
Production script engines need debuggability: line-level errors, variable inspection, and ideally a DAP (Debug Adapter Protocol) endpoint that VS Code can connect to. Lua 5.4 exposes a full debug API; AngelScript ships a debugger helper class.
// Lua: install a debug hook to log every line executed
lua_sethook(L, [](lua_State* L, lua_Debug* ar) {
lua_getinfo(L, "nSl", ar);
std::println(" [{}:{}] {}", ar->source, ar->currentline,
ar->name ? ar->name : "?");
}, LUA_MASKLINE, 0);
// Better: structured error with traceback on lua_pcall failure
static int traceback_handler(lua_State* L) {
const char* msg = lua_tostring(L, 1);
luaL_traceback(L, L, msg, 1); // append stack trace to message
return 1;
}
// Push handler before the function and use lua_pcall with msgh
lua_pushcfunction(L, traceback_handler); // stack: [handler]
lua_getglobal(L, "my_function"); // stack: [handler, fn]
int status = lua_pcall(L, 0, 0, -2); // -2 = handler position
if (status != LUA_OK) {
std::println(stderr, "{}", lua_tostring(L, -1));
lua_pop(L, 1);
}
lua_pop(L, 1); // pop handler
// Expose source info to error messages (embed source name in script)
luaL_loadbuffer(L, source_code.c_str(), source_code.size(),
"@game_logic.lua"); // '@' prefix → shown as filename in errors
// AngelScript: built-in CDebugger helper
#include <scriptstdstring/scriptstdstring.h>
// CDebugger debugger;
// debugger.SetEngine(engine);
// ctx->SetLineCallback(asMETHOD(CDebugger, LineCallback), &debugger, asCALL_THISCALL);
// Then call debugger.TakeCommands(ctx) in the line callback to pause execution| Technique | Lua | AngelScript | QuickJS |
|---|---|---|---|
| Line-level hook | lua_sethook(LUA_MASKLINE) | SetLineCallback | JS_SetInterruptHandler |
| Stack traceback | luaL_traceback | asIScriptContext::GetCallstackSize | JS_GetStack |
| Remote debugger | MobDebug | CDebugger (shipped) | Chrome DevTools protocol |
| Source maps | @filename prefix in luaL_loadbuffer | Module sections | Built-in |