Huge and complex situation there.
[code]#
A fatal error has been detected by the Java Runtime Environment:
EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x721e0693, pid=3780, tid=6232
JRE version: Java™ SE Runtime Environment (7.0_60-b19) (build 1.7.0_60-b19)
Java VM: Java HotSpot™ Client VM (24.60-b09 mixed mode windows-x86 )
Problematic frame:
V [jvm.dll+0x140693]
Failed to write core dump.
If you would like to submit a bug report, please visit:
#[/code]
This happens just after the client jumps from the starting to the main project page. This one runs some scripts to load the project context.
After this error the gateway reboots, but the strangest thing is since that, the server machine resets (java exception: connection reset) every connection coming from the client IP, directed to any port (not only ignition port), and the even rebooting the machine, it continues to reset any connection
On the other side, all the machine services are still available to other clients, coming from other IPs (of course, until they load the magic page).
I really don’t know how to study and solve this issue.
This came after some minor updates. The machine has very low CPU/RAM/DISK usage.
Attached the error pid file
Enlarging the gateway RAM and removing some bindings
(like “if(isAlarmActive(”/*"),“Please Check Anomalies”,“All Systems OK”)")
the problem seems solved
again critical exceptions:
INFO | jvm 1 | 2014/06/30 09:35:49 | INFO [LocalDBManagerImpl ] [09:35:49,299]: Creating auto-backup of internal database "settings"...
INFO | jvm 1 | 2014/06/30 09:35:49 | INFO [ENGINE ] [09:35:49,300]: checkpointClose start
INFO | jvm 1 | 2014/06/30 09:35:49 | INFO [ENGINE ] [09:35:49,362]: dataFileCache commit start
INFO | jvm 1 | 2014/06/30 09:35:49 | INFO [ENGINE ] [09:35:49,487]: checkpointClose end
INFO | jvm 1 | 2014/06/30 09:35:49 | INFO [ENGINE ] [09:35:49,488]: Initiating backup of instance 'settings'
INFO | jvm 1 | 2014/06/30 09:35:50 | 1 / 4 settings.properties... ERROR [LocalDBManagerImpl ] [09:35:49,946]: Cannot create autobackup for internal database "settings".
INFO | jvm 1 | 2014/06/30 09:35:50 | java.sql.SQLException: java.lang.OutOfMemoryError
INFO | jvm 1 | 2014/06/30 09:35:50 | at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | at org.hsqldb.jdbc.JDBCStatement.fetchResult(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | at org.hsqldb.jdbc.JDBCStatement.executeUpdate(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | at org.apache.commons.dbcp.DelegatingStatement.executeUpdate(DelegatingStatement.java:228)
INFO | jvm 1 | 2014/06/30 09:35:50 | at com.inductiveautomation.ignition.gateway.localdb.hsql.HSQLDBManager$AutoBackupDaemon.run(HSQLDBManager.java:618)
INFO | jvm 1 | 2014/06/30 09:35:50 | at com.inductiveautomation.ignition.common.execution.impl.BasicExecutionEngine$TrackedTask.run(BasicExecutionEngine.java:573)
INFO | jvm 1 | 2014/06/30 09:35:50 | at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | at java.lang.Thread.run(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | Caused by: org.hsqldb.HsqlException: java.lang.OutOfMemoryError
INFO | jvm 1 | 2014/06/30 09:35:50 | at org.hsqldb.error.Error.error(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | at org.hsqldb.result.Result.newErrorResult(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | at org.hsqldb.StatementCommand.execute(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | at org.hsqldb.Session.executeCompiledStatement(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | at org.hsqldb.Session.executeDirectStatement(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | at org.hsqldb.Session.execute(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | ... 12 more
INFO | jvm 1 | 2014/06/30 09:35:50 | Caused by: java.lang.OutOfMemoryError
INFO | jvm 1 | 2014/06/30 09:35:50 | at java.io.FileInputStream.readBytes(Native Method)
INFO | jvm 1 | 2014/06/30 09:35:50 | at java.io.FileInputStream.read(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | at org.hsqldb.lib.InputStreamWrapper.read(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | at org.hsqldb.lib.InputStreamWrapper.read(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | at org.hsqldb.lib.tar.TarGenerator$TarEntrySupplicant.write(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | at org.hsqldb.lib.tar.TarGenerator.write(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | at org.hsqldb.lib.tar.DbBackup.write(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | at org.hsqldb.persist.Logger.backupInternal(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | at org.hsqldb.persist.Logger.backup(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | at org.hsqldb.StatementCommand.getResult(Unknown Source)
INFO | jvm 1 | 2014/06/30 09:35:50 | ... 16 more
Based on the logs you just posted, looks like still having memory issues on the gateway. Have you checked task manager to see what the Java.exe memory usage looks like? With message you are seeing it looks like the JVM for the gateway is running out of memory and that is preventing it from doing its normal backup.
java.exe is in the correct mem range.
java = about 700 MB
range = 1 GB min - 3 GB max
Some of the configured devices are down for some hours during the day, this is normal.
I have many offline and timeout messages in the console. Can i have any issue with this and the modbus2 driver?
Thank you
How often is it locking up and what are the hardware specifications for the Machine? In the logs it looks like you are running a 32 bit version of Java is this correct ? If so then this might explain what the project is freezing, because you have the JVM max set to 3 gb. With a 32 bit JVM its max size can only grow to 1.5 gb which is only half of the configured Max.
I’ve upgraded to Ignition 7.6, Java 8 and now i can set mem to 3 GB.
Thank you.