Hello MicroEJ team,
I am developing a VEE Port with an architecture 8.1.0 and would like to support the Multi-Sandbox mode.
I implemented the LLKERNEL_IMPL functions and passed the kernel tests (VEEPortQualificationTools/tests/llkernel at master · MicroEJ/VEEPortQualificationTools · GitHub) successfully.
I then used the Kernel GREEN (GitHub - MicroEJ/Kernel-GREEN: This repository contains a Kernel code template that offers basic services for developing Sandboxed Applications.) which is running without issue.
Finally I created a Sandboxed Application Project from my workspace (SDK Dist. 23.07, MicroEJ SDK 5.8.2) to test the feature installation on the kernel. I didn’t modify the code so the app should just print the auto generated messages when starting or stopping it.
I imported the VDE GREEN in my workspace, created a launcher by selecting this VDE and used the settings “Local Deployment (Socket)”, however I get the following error:
installcommand INFO: Receiving an application: MyFeature - use storage: true
installcommand INFO: Storing the application MyFeature
installcommand INFO: Application storage done
installcommand INFO: Installing the application MyFeature
installcommand SEVERE: code=58
Exception in thread "Rcommand Admin /192.168.1.10:51505" ej.kf.InvalidFormatException: code=58
at java.lang.System.getStackTrace:0x832bc@
at java.lang.Throwable.fillInStackTrace(line:82)
at java.lang.Throwable.<init>(line:37)
at java.lang.Exception.<init>(line:18)
at com.is2t.kf.ExceptionWithErrorCode.<init>(line:28)
at com.is2t.kf.ExceptionWithErrorCode.<init>(line:24)
at ej.kf.InvalidFormatException.<init>(line:30)
at com.is2t.kf.FoLoader.getFeatureCodeChunkBlock(line:328)
at com.is2t.kf.FoLoader.newProgAllocationSection(line:298)
at com.is2t.kf.FoLoader.loadAndProcessSection(line:640)
at com.is2t.kf.FoLoader.load(line:532)
at com.is2t.kf.FoLoader.loadFO(line:217)
at com.is2t.kf.FoFeatureLoader.load(line:76)
at ej.kf.Kernel.install(line:170)
at com.microej.library.kernel.rcommand.local.InstallCommand.loadAndInstallFeature(line:146)
at com.microej.library.kernel.rcommand.local.InstallCommand.commandReceived(line:72)
at com.microej.library.kernel.rcommand.local.AbstractCommand.commandReceived(line:32)
at ej.rcommand.impl.DefaultRemoteCommandManager$CommandReader.run(line:91)
at java.lang.Thread.run(line:311)
at java.lang.Thread.runWrapper(line:464)
at java.lang.Thread.callWrapper(line:449)
The kernel is still running by the app wasn’t installed. What is the cause of this exception?
Regards,
Philip
I forgot to mention that it works on the simulator:
=============== [ Initialization Stage ] ===============
=============== [ Converting fonts ] ===============
=============== [ Converting images ] ===============
=============== [ Launching on Simulator ] ===============
kernel INFO: Kernel startup
kernel INFO: Initializing custom Security Manager
kernel INFO: Registering mandatory services
kernel INFO: Registering featureStateListener
kernel INFO: Loading and installing features from the storage (FS)
kernel INFO: Starting NTP and CommandServer
kernel INFO: Start the ntp client
kernel INFO: Use the connectivity manager
kernel INFO: Start the admin server
Feature MyFeature started!
kernel INFO: State update: MyFeature (v0.1.0) has changed from state INSTALLED to STARTED
remotecommandserver INFO: Server listening on port 4000
Hello @p.calamarino ,
The error code means that the Feature cannot be installed because the code chunk size of your Kernel is too small to accept the .fo file.
Please double-check that your .fo file has been built for the Kernel on which you are trying to install it. For that, verify that the Kernel uid in the Feature information file is the same than the Kernel uid in the Kernel information file.
See SOAR Output Files — MicroEJ Documentation.
–Frédéric
Hello,
My kernel and my feature have the same kernel uid (see screenshot attached).
Regards
Everything looks good.
However, by looking to the source code, it seems there is a code section in your .fo file that is greater than the Feature code chunk size (aka 65536 bytes).
To go further, please dump the sections of the .fo file using the command readelf -WS application.fo
You should get a dump similar to the one described here:
There are 8 section headers, starting at offset 0x34:
Section Headers:
[Nr] Name Type Addr Off Size ES Flg Lk Inf Al
[ 0] NULL 00000000 000000 000000 00 0 0 0
[ 1] .soar.rel LOPROC+0 00000000 000174 000bcc 00 6 0 4
[ 2] .strtab STRTAB 00000000 000d40 000063 00 0 0 1
[ 3] .symtab SYMTAB 00000000 000da4 000050 10 2 1 4
[ 4] .bss.soar.feature NOBITS 00000000 000df4 000050 00 A 0 0 4
[ 5] .rodata.microej.resources PROGBITS 00000000 000e00 079080 00 A 0 0 64
[ 6] .rodata PROGBITS 00000000 079e80 001974 00 A 0 0 16
[ 7] .shstrtab STRTAB 00000000 07b7f4 000059 00 0 0 1
Here is the output:
$ readelf -WS application.fo
There are 9 section headers, starting at offset 0x34:
Section Headers:
[Nr] Name Type Addr Off Size ES Flg Lk Inf Al
[ 0] NULL 00000000 000000 000000 00 0 0 0
[ 1] .strtab STRTAB 00000000 00019c 000046 00 0 0 1
[ 2] .symtab SYMTAB 00000000 0001e4 000040 10 1 1 4
[ 3] .bss.soar.feature NOBITS 00000000 000224 00006c 00 A 0 0 4
[ 4] .kernel.uid STRTAB 00000000 000224 000018 00 0 0 0
[ 5] .rodata.microej.resources PROGBITS 00000000 00023c 000000 00 A 0 0 4
[ 6] .rodata PROGBITS 00000000 000240 0005dc 00 A 0 0 16
[ 7] .shstrtab STRTAB 00000000 00081c 000065 00 0 0 1
[ 8] .soar.rel LOPROC+0 00000000 000884 0002d9 00 6 0 4
Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
L (link order), O (extra OS processing required), G (group), T (TLS),
C (compressed), x (unknown), o (OS specific), E (exclude),
D (mbind), p (processor specific)
We can see there is no section greater than 65536 bytes.
I suspect now that the InputStream given to Kernel.install() returns corrupted bytes compared to the original .fo file.
Can you give me more details on how the .fo file is transfered to the device ? what kind of InputStream ? what is the implementation of the InputStream in the VEE Port ?
I don’t have much details except that I used the local deploy settings in my launcher.
If I trace back the calls to Kernel.install() it is called like this:
private Feature loadAndInstallFeature(String featureToken)
throws IOException, InvalidFormatException, IncompatibleFeatureException, AlreadyLoadedFeatureException {
try (InputStream storedFeatureStream = this.applicationStorage.load(featureToken)) {
Feature feature = Kernel.install(storedFeatureStream);
The stream seems to come from the storage:
if (useStorage) {
this.logger.info("Storing the application " + applicationName); //$NON-NLS-1$
String featureToken = this.applicationStorage.store(stream);
this.logger.info("Application storage done"); //$NON-NLS-1$
this.logger.info("Installing the application " + applicationName); //$NON-NLS-1$
Feature feature = loadAndInstallFeature(featureToken);
All of this is done in the local-deploy-impl library.
OK so if the .fo file is stored on the device, then the next step is to verify the .fo stored on the device is not corrupted.
Please find a way to retrieve the file from the storage of the device. Then you will be able to compare the file content on your workstation; for example you can use sha1sum linux command.
My idea behind that is to understand if the file is corrupted between the transfer and the storage on the device or between the load from the device and the call to Kernel.install().
I had to modify a bit of Java in the kernel because the file is actually deleted if there is an exception.
Here is the sha1 on the target:
# sha1sum /home/microej/storage/kernel/app_181419297
7336e52cec3e17da26a6e8db6d177680cc11acb3 /home/microej/storage/kernel/app_181419297
And the .fo in my workspace:
$ sha1sum application.fo
7336e52cec3e17da26a6e8db6d177680cc11acb3 application.fo
So the files are identical.
I also tried to bypass the storage by disabling “Use persistent storage” in my launcher and the result is the same.
Ok so the .fo file is valid on the filesystem device. We have to trace the bytes that are read by Kernel.install().
For that, I just prepared a dedicated InputStream named DebugInputStream. This stream will print the bytes on the standard output when they are consumed.
0x7f
0x45
0x4c
0x46
0x01
0x01
0x01
...
- Copy the standard output traces to a file. Let’s call it:
traceFODevice.txt
I also prepared in the same class a main entry point to dump the bytes of the original .fo file on your workstation. It can be run as a standard J2SE project using the following command:
java -cp [path_to_kernel_project]/bin com.microej.example.DebugInputStream [path_to_fo_file] > traceFOWorkstation.txt
Once you get here, you can compare the content of the two files and show me the diff.
Appendix
Here is the DebugInputStream class:
package com.microej.example;
import java.io.FileInputStream;
import java.io.FilterInputStream;
import java.io.IOException;
import java.io.InputStream;
/**
* A {@link FilterInputStream} that prints the bytes on the standard output when they are consumed.
*/
public class DebugInputStream extends FilterInputStream {
public DebugInputStream(InputStream in) {
super(in);
}
@Override
public int read() throws IOException {
int res = this.in.read();
if (res != -1) {
printHex(res);
}
return res;
}
@Override
public int read(byte[] b) throws IOException {
int res = this.in.read(b);
if (res != -1) {
for (int i = 0; i < res; ++i) {
printHex(b[i] & 0xFF);
}
}
return res;
}
@Override
public int read(byte[] b, int off, int len) throws IOException {
int res = this.in.read(b, off, len);
if (res != -1) {
for (int i = 0; i < res; ++i) {
printHex(b[off + i] & 0xFF);
}
}
return res;
}
@Override
public long skip(long n) throws IOException {
long res = super.skip(n);
System.out.println("Skipped :" + res);
return res;
}
private void printHex(int res) {
System.out.print("0x");
if (res < 16) {
System.out.print('0');
}
System.out.println(Integer.toHexString(res));
}
/**
* This main entry point prints the content of the file on standard output.
*/
public static void main(String[] args) throws IOException {
String name = args.length == 0 ? null : args[0];
if (name == null) {
System.out.println("[ERROR] Missing filename as first argument.");
return;
}
try (FileInputStream fis = new FileInputStream(name)) {
DebugInputStream dis = new DebugInputStream(fis);
int read;
while ((read = dis.read()) != -1) {
}
}
}
}
It seems that the exception is thrown after two skip:
--- traceFODevice.txt 2024-02-23 09:39:00.776666000 +0100
+++ traceFOWorkstation.txt 2024-02-23 09:39:59.661870200 +0100
@@ -480,7 +480,8 @@
0x74
0x00
0x00
-Skipped :2
+0x00
+0x00
0x45
0x00
0x00
@@ -569,4 +570,2340 @@
0xeb
0xdf
0x85
-Skipped :4
+0x00
+0x00
+0x00
+0x00
+0xf0
+0xff
0xeb
0xdf
0x85
Skipped :4
installcommand SEVERE: code=58
Exception in thread "Rcommand Admin /192.168.2.111:55231" ej.kf.InvalidFormatException: code=58
Rest is just the remaining of the stream from workstation file.
tracediff.txt (16.2 KB)
Thanks so we can confirm that the .fo read is not corrupted.
Looking back to the code when the exception is thrown:
I suspect now that the Feature code chunk section in RAM could be corrupted, consequently allocatedBlock.getSize() would not return the expected value (65535).
Hello Frederic,
After investigation I found that one of my native call may have reset to zero that part of the RAM during the installation. Now that I fixed this issue I no longer have the exception and the installation completes successfully.
Thank you for the help!
