Skip to content

Added Lua compat53: fixed the non-interactive mode: added .gitmodules to build.yml#6

Merged
BanceDev merged 33 commits intoBanceDev:mainfrom
ItsMeForLua:my-feature-branch
Aug 19, 2025
Merged

Added Lua compat53: fixed the non-interactive mode: added .gitmodules to build.yml#6
BanceDev merged 33 commits intoBanceDev:mainfrom
ItsMeForLua:my-feature-branch

Conversation

@ItsMeForLua
Copy link
Contributor

The linter issue has been fixed.
Here is the complete pull request without all the noise as was the issue previously.


Relevance:

(lush/pull/4)

(lush/pull/5)

Feature Request: Bundle or Preload lua-compat-5.3 for Lua 5.1 Compatibility · Issue #3 · BanceDev/lush · GitHub

@BanceDev
Copy link
Owner

I'll take a look at this later today, and hopefully we can get stuff merged

@BanceDev
Copy link
Owner

couple notes so far.

  1. the .gitmodules doesnt seem to be doing anything so far. the old hashmap code is still being used and it looks like the files for the compat are just copied in. I don't mind the hashmap just being the old code but the module ur using is fine as well. I just dont want to copy in dependencies to the main repo if possible.

  2. I tried doing some quick testing of my own with compatibility with some 5.2 features and it seems to not be working. Here's the script I ran:

-- Lua 5.2-specific features
local _ENV = { print = print, x = 123 } -- lexical environments
function show_x()
    print("x =", x)
end

show_x()

-- Bitwise ops (added in 5.2)
local a, b = 0x5, 0x3
print("Bitwise AND:", a & b)

-- load() replaces loadstring() (binary and text)
local f = load("return 10 + 20")
print("Loaded result:", f())

-- table.pack / table.unpack
local t = table.pack(1, 2, 3, nil, 5)
print("Packed length:", t.n)
print("Unpacked values:", table.unpack(t))

Here's the output:

[lanceb@archlinux: ~] ./test_52.lua 
x =	123
Bitwise AND:	1
[C] Error executing script: ./test_52.lua:14: attempt to call a nil value (global 'load')

I'm not certain what may be causing this or if I may even have a specific issue on my end. Do you get the same output when running this script in your build of the shell?

@ItsMeForLua
Copy link
Contributor Author

ItsMeForLua commented Jul 20, 2025

couple notes so far.

  1. the .gitmodules doesnt seem to be doing anything so far. the old hashmap code is still being used and it looks like the files for the compat are just copied in. I don't mind the hashmap just being the old code but the module ur using is fine as well. I just dont want to copy in dependencies to the main repo if possible.
  2. I tried doing some quick testing of my own with compatibility with some 5.2 features and it seems to not be working. Here's the script I ran:
-- Lua 5.2-specific features
local _ENV = { print = print, x = 123 } -- lexical environments
function show_x()
    print("x =", x)
end

show_x()

-- Bitwise ops (added in 5.2)
local a, b = 0x5, 0x3
print("Bitwise AND:", a & b)

-- load() replaces loadstring() (binary and text)
local f = load("return 10 + 20")
print("Loaded result:", f())

-- table.pack / table.unpack
local t = table.pack(1, 2, 3, nil, 5)
print("Packed length:", t.n)
print("Unpacked values:", table.unpack(t))

Here's the output:

[lanceb@archlinux: ~] ./test_52.lua 
x =	123
Bitwise AND:	1
[C] Error executing script: ./test_52.lua:14: attempt to call a nil value (global 'load')

I'm not certain what may be causing this or if I may even have a specific issue on my end. Do you get the same output when running this script in your build of the shell?

I'll get back to you on either Wednesday or thursday.
(1) I did end up bundling the dependencies in the repo apparently, so I'll remove those, and rely on the auto-install.

(2) Your situation could be a "it works on my machine" kind of ordeal. It did work on my machine, but, that could be because I happened to set up the environment just right for it to work on my particular system.
I do know that the compat53 library doesn't bring compatibility for EVERY legacy function, but it does make at-least 80% of them compatible.
The issue could also be that I didn't set up the dependency automation as well as I thought I did, leading to an incomplete or bugged compat library on your machine.

So,
I'll write a dockerfile and do some testing. But i have a few essays I need to work on for school, so it wont be immediate.


EDIT: It just clicked that "it works on my machine" can come across badly. Apologies if it did! I absolutely meant it in a neutral "we have different environments to figure out" way.

… library (libm); removed the incorrect luaopen_compat53 call that was still present
…in to run the test, we'll do the entire compile and test sequence within a temporary container. Even though it copied the bin directory to the Jenkins workspace, the docker run command with the volume mount (-v) isn't finding it.
…pat53 library actually provides: the compat53 library is designed to bring Lua 5.1/5.2 up to the API level of 5.3, but it doesn't add new syntax to the parser, such as the & bitwise operator.: ln:25-47
…ibrary is already loaded and available in the global space for scripts running within lush. The test script in the Jenkinsfile just needs to call it directly without using require: ln:27-65: also removed gemini's weird copout at ln:38-39; wont be using gemini anymore for this boilerplate, since it's trying to (and failing to...) 'troubleshoot' for me, unprompted.
…TLIB compiler definition. This flag instructs the compat-5.3 submodule to build the actual bit32 library instead of a stub that throws a deprecated error.

- The main function in src/lush.c is modified to explicitly load the bit32 and utf8 libraries into the global Lua state at startup using luaL_requiref. This makes them directly accessible to all scripts running in the shell, which is necessary for the test script to find and use the bit32 functions without a require() call.
- Added a null check after luaL_newstate() in src/lush.c to ensure the Lua state is created successfully before its actually used.
@ItsMeForLua
Copy link
Contributor Author

ItsMeForLua commented Jul 23, 2025

@BanceDev
NOTE: I'm keeping everything within this pull request this time, rather than making a whole new pull, simply because it's much easier to see commit history, and the context TO those commits.


REVIEW: CI/CD Pipeline + Lua Compatibility Improvements

1. Jenkins Pipeline

  • Added a complete Jenkinsfile to my feature branch
  • Updated Jenkins test scripts to work with the globally available bit32 library
  • Moved the entire compile and test sequence inside temporary containers to fix Docker volume mount issues

2. Docker

  • Self explanatory

3. Lua Compat53 Integration

  • Updated premake5.lua to include LUA_COMPAT_BITLIB compiler definition, making compat-5.3 build the actual bit32 library instead of a deprecated stub
  • Added .gitmodules initialization to the install.sh script

4. Lua State Initialization

  • Modified src/lush.c to explicitly load bit32 and utf8 libraries globally at startup using luaL_requiref
  • Added simple error handling: implemented null check after luaL_newstate()
  • Made libraries directly accessible without requiring require() calls in scripts

Jenkins Output:

...

==== Building lush (debug) ====
Creating obj/Debug
builtins.c
compat-5.3.c
hashmap.c
history.c
lbitlib.c
liolib.c
lstrlib.c
ltablib.c
lua_api.c
lush.c
lutf8lib.c
Linking lush
--- Running Lua 5.2 Compatibility Test ---
Basic print test: Hello from Lush!
Bitwise AND of	5	and	3	=	1
Bitwise OR of	5	and	3	=	7
Loaded function result:	30
Unpacked values:	1	2	3
String length:	13
Substring:	Hello
Math operations:
  sqrt(16) =	4.0
  max(10, 20, 5) =	20
--- Test Complete: All basic features working ---

...
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

Final Notes:
The Docker/Jenkins infrastructure was necessary for me to properly test the changes. The containerized build process ensures the compat53 library(the shims) integration works correctly across different environments.
I do acknowledge that github actions exists (and is used for this repo), however, in-order to prevent you from having to do more work due to any of my silly mistakes, and in order for me to make sure the code is all completely sound, I had added the jenkins pipeline as a "pre-github CI/CD" so-to-speak.

I think this would also make future pull requests MUCH more convenient since, testing can be done locally BEFORE submitting a pull request.

If you want to include these Docker and Jenkins files in the main repository, we might consider adding instructions on how to use them, possibly in a dev-notes file. However, that could be discussed separately.

EDIT:
Also, it seems in one of my previous pulls, I had replaced your custom hashmaps with https://github.com/tidwall/hashmap.c.git
I noticed it in the .gitmodules file.
Let me know if you're actually comfortable with that decision, or if you want to continue using your custom hashmaps.

…ding the populated lib/ directory from the submodules) into the running container, ensuring all the required source files are available for the build process.
…lly automating the module initialization via jenkins: DEBUG VERSION: Jenkinsfile
…nd thus, will init .gitmodules: ln:10 removed from original Jenkinsfile
…at is missing its dependencies in the lib/ directory, leading to a 'No rule to make target' error: updated Dockerfile to remove the submodule command at: ln:29: and: Jenkinsfile will use the git submodule sync command. This will ensure the agent's repository is correctly configured before fetching the submodule code: at: ln:9-17
@ItsMeForLua
Copy link
Contributor Author

ItsMeForLua commented Jul 23, 2025

#3f4e2c5
#12a94cd
#8d9b53f

I will be moving the pipeline files to a seperate PR, as attempting to fix the gitmodule initilization command from the Jenkins pipeline inside the docker container is causing this PR to be quite large.

The core functionality is already proven, the issue with the pipeline has no effect on the end-user product.
I do not want to burden you will these CI commits!

@ItsMeForLua
Copy link
Contributor Author

-- Lua 5.2-specific features
local _ENV = { print = print, x = 123 } -- lexical environments
function show_x()
    print("x =", x)
end

show_x()

-- Bitwise ops (added in 5.2)
local a, b = 0x5, 0x3
print("Bitwise AND:", a & b)

-- load() replaces loadstring() (binary and text)
local f = load("return 10 + 20")
print("Loaded result:", f())

-- table.pack / table.unpack
local t = table.pack(1, 2, 3, nil, 5)
print("Packed length:", t.n)
print("Unpacked values:", table.unpack(t))

Here's the output:

[lanceb@archlinux: ~] ./test_52.lua 
x =	123
Bitwise AND:	1
[C] Error executing script: ./test_52.lua:14: attempt to call a nil value (global 'load')

I tried running your script:

Since, in Lua 5.2+, when you declare a local variable named _ENV, you are creating a brand new, sandboxed environment for that block of code. Your new environment contains only what you explicitly put inside it. In this case, it only contains the print function and the variable x.

When your script later tries to call load(), table.pack(), or use the & operator, it looks for them in your tiny, custom _ENV. Since they don't exist there, it finds nil, which leads to the error: attempt to call a nil value (global 'load').

It took me some time to figure out why that wasn't working!


-- NOTE: I had gemini generate this version
-- Create a new table for our custom environment
local my_env = {
  x = 123
}

-- Set the new environment's metatable to inherit from the global environment (_G)
-- This gives it access to 'print', 'load', 'table', etc.
setmetatable(my_env, { __index = _G })

-- Apply this new environment to the current script
-- (This handles both Lua 5.2+ and 5.1 for full compatibility)
if _ENV then
    _ENV = my_env
else
    setfenv(1, my_env)
end


-- Now the rest of the script will work correctly!

function show_x()
    print("x =", x) -- 'x' is from our custom env
end

show_x()

-- Bitwise ops will work because the metamethods are in _G
local a, b = 5, 3
print("Bitwise AND:", a & b)

-- 'load' will be found in _G
local f, err = load("return 10 + 20")
if f then
    print("Loaded result:", f())
else
    print("Error loading string:", err)
end

-- 'table' will be found in _G
local t = table.pack(1, 2, 3, nil, 5)
print("Packed length:", t.n)
print("Unpacked values:", table.unpack(t))

Output:

[arandreschool@Johnny: ~/misc/testingstuff] ./test_52.lua
x =     123
Bitwise AND:    1
Loaded result:  30
Packed length:  5
Unpacked values:        1       2       3       nil     5
[arandreschool@Johnny: ~/misc/testingstuff]

@BanceDev
Copy link
Owner

hey I'm sorry this took me forever to get back to but I ended up testing everything and it looks good, gonna get this merged and staged for the next release. Thanks for all of your work :)

@BanceDev BanceDev merged commit 810c1f8 into BanceDev:main Aug 19, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants