serializeToBundle issue

[habilmohammed]

Hi,
I am trying to serialize a simple spark model using mleap serializeToBundle method, but for some reason its failing with the below attached error, that the spark version is not supported. I have tried all the combinations of mleap and spark version as mentioned in the mleap doc, but the error remains the same.

Any help/suggestions on the resolving this would be appreciated.

python: 3.8
spark version 3.3.0
mleap: 0.22.0

data = spark.read.format("libsvm").load("sample_libsvm_data.txt")
featureIndexer = VectorIndexer(inputCol="features", outputCol="indexedFeatures", maxCategories=4).fit(data)
(trainingData, testData) = data.randomSplit([0.7, 0.3])
dt = DecisionTreeRegressor(featuresCol="indexedFeatures")
pipeline = Pipeline(stages=[featureIndexer, dt])
model = pipeline.fit(trainingData)

model.serializeToBundle("jar:file:/pyspark.example.zip", model.transform(data))

2024-07-22 17:29:21.070 IST
: java.lang.ExceptionInInitializerError
2024-07-22 17:29:21.070 IST
	at ml.combust.mleap.spark.SimpleSparkSerializer$$anonfun$1.apply(SimpleSparkSerializer.scala:17)
2024-07-22 17:29:21.070 IST
	at ml.combust.mleap.spark.SimpleSparkSerializer$$anonfun$1.apply(SimpleSparkSerializer.scala:17)
2024-07-22 17:29:21.070 IST
	at scala.Option.map(Option.scala:230)
2024-07-22 17:29:21.070 IST
	at ml.combust.mleap.spark.SimpleSparkSerializer.serializeToBundle(SimpleSparkSerializer.scala:17)
2024-07-22 17:29:21.070 IST
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2024-07-22 17:29:21.070 IST
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2024-07-22 17:29:21.070 IST
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2024-07-22 17:29:21.070 IST
	at java.lang.reflect.Method.invoke(Method.java:498)
2024-07-22 17:29:21.070 IST
	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
2024-07-22 17:29:21.070 IST
	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
2024-07-22 17:29:21.070 IST
	at py4j.Gateway.invoke(Gateway.java:282)
2024-07-22 17:29:21.070 IST
	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
2024-07-22 17:29:21.070 IST
	at py4j.commands.CallCommand.execute(CallCommand.java:79)
2024-07-22 17:29:21.070 IST
	at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
2024-07-22 17:29:21.070 IST
	at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
2024-07-22 17:29:21.070 IST
	at java.lang.Thread.run(Thread.java:750)
2024-07-22 17:29:21.070 IST
Caused by: java.lang.IllegalStateException: unsupported Spark version: 3.3.0
	at org.apache.spark.ml.bundle.SparkBundleContext$.<init>(SparkBundleContext.scala:23)
	at org.apache.spark.ml.bundle.SparkBundleContext$.<clinit>(SparkBundleContext.scala)
	... 16 more