Running 1.2.1 on clojars, curious if it's possible to send forms for distributed eval

[joinr]

I know there is a 1.6 snapshot, but it's not published currently. This problem may be changed by now.

Since we can send arbitrary functions, why can't I send something like...

(partial eval '(defn say [arg] (println arg)))

?

This is serializable. It should be interpreted on the cluster instance. Basically should provide a means to patch things at runtime and distribute stuff over the cluster. As it stands, nothing seems to happen. Certainly no changes to the namespace, but no errors either. I do get errors if I send a repl-generated function, like (fn [] ...) as a Runnable, since the classloader complains. Again, this could be a function of older versions though.

In principle it seems like this should be possible, and it would add a degree of flexibility (e.g. I don't have to AOT compile everything and distribute as a dependency to ensure identical classes, etc.). I guess the other option is to have a simple function/message handler that uses ns-resolve to find the function and the like.

Just curious if I am running against architectural constraints or not.

[joinr]

I actually just noticed I am consistent with the current master (I thought I saw a more current snapshot, but was in error).

[joinr]

lexically scoped stuff works, e.g.:

(ask (partial eval '(let [f (fn [x] (+ x 1))] (println (f 1)))) :members :all)

[tolitius]

hey @joinr can you share the use case of what it is you need to do?
sending eval over the cluster does work. the reason you don't see anything is because, in this example eval does not do anything side effectful that you can see happening on the cluster:

boot.user=> (eval '(defn say [arg] (println arg)))
#'boot.user/say
boot.user=> (partial eval '(defn say [arg] (println arg)))
#object[clojure.core$partial$fn__5839 0x63ca4c68 "clojure.core$partial$fn__5839@63ca4c68"]

boot.user=> (task (eval '(defn say [arg] (println arg))))
nil
boot.user=> (task (partial (eval '(defn say [arg] (println arg)))))
nil

i.e. the result is the same as in local REPL: nothing is printed.

eval does work otherwise:

=> (task #(eval '(do (Thread/sleep 2000) (println "printing 42 remotely"))))
nil
boot.user=>
... in 2 seconds on the cluster
printing 42 remotely

[joinr]

the reason you don't see anything is because, in this example eval does not do anything side effectful that you can see happening on the cluster:

If this eval worked, then the server processes running on the cluster (e.g. the same process I am in running the REPL from, which should be evaluating the forms) should have the side-effect of invoking defn and mutating the namespace (at least I think they would). It appears not to. Or perhaps the evaluation is occuring somewhere else, or under some restrictions that prevent forms like def from taking effect. I am reading up on hazelcast's internals, but I currently have limited knowledge on the actual evaluation model.

The use case is simple hot-loading or patching running code as I am developing. I would like to push updates and have cluster members leverage clojure's dynamism to update/load code at runtime.

Something like this:

user=> (future (defn blah [x] (+ x 2)))
#object[clojure.core$future_call$reify__8479 0x4a11eb84 {:status :pending, :val nil}]
user=> (blah 2)
4

works. Replace future with task and I would hope for similar semantics.

[joinr]

I revisited this recently; funny enough it looks like eval is happening, except its in the clojure.core namespace. So if we send along in-ns as our form to eval, it works:

hazeldemo.core> (ch/task (partial eval '(in-ns 'hazeldemo.core) (defn hello [] (println "hello"))))
nil
hazeldemo.core> hello
#function[hazeldemo.core/eval15374/hello--15375]
hazeldemo.core> (ch/task (partial hello))
hello ;;also hello on remote machines in cluster

[joinr]

Ah, looks like remote machines have the hello function still unbound (but it's a var in the right namespace....)

[joinr]

(defn interpret [args]
  (binding [*ns* this-ns]
    (eval (read-string args))))

on both machines, invoked via task from machine1:

(ch/task (partial interpret "(defn hello [] (println :hello))"))

remote machine2 actually defines the function, but machine 1 does not. if I back fill (on machine1) an implement the function (I thought interpret would've done it but eh...)

(defn hello [] (println :hellome))

Then ch/task invocation of hello is now viable.