Sharing dynamically loaded classes with JShell instance

Question

Please view the edits below

I'm trying to create a JShell instance that gives me access to, and lets me interact with objects in the JVM it was created in. This works fine with classes that have been available at compile time but fails for classes that are loaded dynamically.

public class Main {

    public static final int A = 1;
    public static Main M;

    public static void main(String[] args) throws Exception {
        M = new Main();
        ClassLoader cl = new URLClassLoader(new URL[]{new File("Example.jar").toURL()}, Main.class.getClassLoader());
        Class<?> bc = cl.loadClass("com.example.test.Dynamic");//Works
        JShell shell = JShell.builder()
                .executionEngine(new ExecutionControlProvider() {
                    @Override
                    public String name() {
                        return "direct";
                    }

                    @Override
                    public ExecutionControl generate(ExecutionEnv ee, Map<String, String> map) throws Throwable {
                        return new DirectExecutionControl();
                    }
                }, null)
                .build();
        shell.eval("System.out.println(com.example.test.Main.A);");//Always works
        shell.eval("System.out.println(com.example.test.Main.M);");//Fails (is null) if executionEngine is not set
        shell.eval("System.out.println(com.example.test.Dynamic.class);");//Always fails
    }
}

Additionally, exchanging DirectExecutionControl with LocalExecutionControl gives the same results, but I do not understand the difference between the two classes.

How would I make classes that are loaded at runtime available to this JShell instance?

Edit: The first part of this question has been solved, below is the updated source code to demonstrate the second part of the issue

public class Main {

    public static void main(String[] args) throws Exception {
        ClassLoader cl = new URLClassLoader(new URL[]{new File("Example.jar").toURL()}, Main.class.getClassLoader());
        Class<?> c = cl.loadClass("com.example.test.C");
        c.getDeclaredField("C").set(null, "initial");
        JShell shell = JShell.builder()
                .executionEngine(new ExecutionControlProvider() {
                    @Override
                    public String name() {
                        return "direct";
                    }

                    @Override
                    public ExecutionControl generate(ExecutionEnv ee, Map<String, String> map) throws Throwable {
                        return new DirectExecutionControl();
                    }
                }, null)
                .build();
        shell.addToClasspath("Example.jar");
        shell.eval("import com.example.test.C;");
        shell.eval("System.out.println(C.C)"); //null
        shell.eval("C.C = \"modified\";");
        shell.eval("System.out.println(C.C)"); //"modified"
        System.out.println(c.getDeclaredField("C").get(null)); //"initial"
    }
}

This is the expected output, if the JVM and the JShell instance do not share any memory, however adding com.example.test.C directly to the project instead of loading it dynamically changes the results as follows:

shell.eval("import com.example.test.C;");
shell.eval("System.out.println(C.C)"); //"initial"
shell.eval("C.C = \"modified\";");
shell.eval("System.out.println(C.C)"); //"modified"
System.out.println(c.getDeclaredField("C").get(null)); //"modified"

Why is the memory between the JVM and the JShell instance not shared for classes loaded at runtime?

EDIT 2: The issue seems to be caused by different class loaders

Executing the following code in the context of the above example:

System.out.println(c.getClassLoader()); //java.net.URLClassLoader
shell.eval("System.out.println(C.class.getClassLoader())"); //jdk.jshell.execution.DefaultLoaderDelegate$RemoteClassLoader
shell.eval("System.out.println(com.example.test.Main.class.getClassLoader())"); //jdk.internal.loader.ClassLoaders$AppClassLoader

This shows, that the same class, com.example.test.C is loaded by two different classloaders. Is it possible to add the class to the JShell instance without loading it again? If no, why is the statically loaded class already loaded?

Answer 1

The solution is to create a custom LoaderDelegate implementation, that supplies instances of already loaded classes instead of loading them again. A simple example is to use the default implementation, DefaultLoaderDelegate (source) and override the findClass method of its internal RemoteClassLoader

@Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
    byte[] b = classObjects.get(name);
    if (b == null) {
        Class<?> c = null;
        try {
            c = Class.forName(name);//Use a custom way to load the class
        } catch(ClassNotFoundException e) {
        }
        if(c == null) {
            return super.findClass(name);
        }
        return c;
    }
    return super.defineClass(name, b, 0, b.length, (CodeSource) null);
}

To create a working JShell instance, use the following code

JShell shell = JShell.builder()
    .executionEngine(new ExecutionControlProvider() {
        @Override
        public String name() {
            return "name";
        }

        @Override
        public ExecutionControl generate(ExecutionEnv ee, Map<String, String> map) throws Throwable {
            return new DirectExecutionControl(new CustomLoaderDelegate());
        }
    }, null)
    .build();
shell.addToClasspath("Example.jar");//Add custom classes to Classpath, otherwise they can not be referenced in the JShell

Answer 2

only speaking to a small part of this rather substantial question:

Additionally, exchanging DirectExecutionControl with LocalExecutionControl gives the same results, but I do not understand the difference between the two classes

LocalExecutionControl extends DirectExecutionControl and it overrides only invoke(Method method), the bodies of which are ...

local:

    Thread snippetThread = new Thread(execThreadGroup, () -> {
            ...
            res[0] = doitMethod.invoke(null, new Object[0]);
            ...
    });

direct:

    Object res = doitMethod.invoke(null, new Object[0]);

so the difference between the two classes is that direct invokes the method in the current thread, and local invokes it in a new thread. the same classloader is used in both cases, so you'd expect the same results in terms of sharing memory and loaded classes

Answer 3

Now, there is better and easier solution:

package ur.pkg;

import jdk.jshell.JShell;
import jdk.jshell.execution.LocalExecutionControlProvider;

public class TestShell {
    public static int testValue = 5;

    public static void main(String[] args) {
        JShell shell = JShell.builder().executionEngine(new LocalExecutionControlProvider(), null).build();
        TestShell.testValue++;
        System.out.println(TestShell.testValue);
        shell.eval("ur.pkg.TestShell.testValue++;").forEach(p -> {
            System.out.println(p.value());
        });
        System.out.println(TestShell.testValue);

    }

}

Default execution engine is JDI, but u can switch it to local or own.