SeisComP AGPL prerelease for beta testing

News
1

Dear SeisComP community,

After over ten years of development and use of SeisComP3 in many institutions worldwide, we are glad to announce the release of a new SeisComP version with a number of major changes we have been working hard on for you, including a new license arrangement.

Some of the new changes are explained below.

  • The name: With the next generation of SeisComP it is time to drop the 3 in SeisComP3. We decided to continue with version numbers starting from 4.0.0 beta. We opened a poll for you to suggest a name scheme for the forthcoming release (see the bottom of the post).

  • The code changes: The new SeisComP supports Python 3 and can use libqt5. (Python versions before 2.7 are no longer supported; Qt4 is still supported.) The installation directory location has moved. A completely new messaging system, replacing spread, has been developed by gempa GmbH.

  • There are no database schema changes introduced with the new version. You will be able to continue to use the existing database which will greatly ease adopting the new version.

  • The License changes: We moved away from the old SeisComP public license to adopt a standard free software license - the Affero General Public License (AGPL). This will allow all stakeholders to easily use and or contribute to the software package. It removes the requirement to complete a license agreement with us, and the restriction to
    non-commercial use.

  • The Repository: With an open license, we are able to share the first new NG release at https://github.com/SeisComP where you can find the SeisComP software collection distributed among several repositories according to ownership, GFZ, GEMPA and other contributors. We would encourage developers to compile and try out the new version. We would be happy to receive feedback. In case of pull request, please note that you will be asked to sign a Contributor License Agreement (CLA). We will be completing the preparation of pre-compiled binaries for regular users and expect to provide them in a month from now.

  • Documentation: we have reviewed most of the documentation, added explanation of the most important concepts for beginners and provided a number of tutorials for some common activities. We will open a new web site for documentation at http://seiscomp.de at the time of the first official release.

  • The downloader pages: we hope that more and more users in the near future will be installing directly from the public repository. To foster this we provide a dedicated tutorial. After a short feedback period, we will provide binaries for the current Ubuntu, Debian and CentOS Linux distributions, from the new web site when it is operational.

The current SeisComP3 Jakarta code will be maintained until March 2022 in order to accommodate the migration to the newer version in complex operational environments. However there will only be bug fixes and security enhancements made, and no new features will be introduced. We strongly encourage all users in operational
environments to start testing this first beta version as soon as possible.

The community and the forum: the SeisComP forum will remain the central place for the community; we plan only a minor layout change introducing a new logo and related style changes.

The development team: this major change has been implemented jointly by gempa GmbH and GFZ-GEOFON. We take this opportunity to acknowledge all contributors so far and we hope to see more active contributors in the
near future with this new licensing model.

We kindly ask the SeisComP community to help us in finding a new name scheme for the future releases and versions.

We kindly encourage you to vote for the new name scheme by Wednesday 22 April 2020. Please participate and contribute!

  • no name, just version number
  • rocks and minerals
  • seismologists and geophysicists
  • Earth discontinuities
  • flowers
  • animals
  • objects of our solar system
  • geographic or geologic expressions
  • vintage software/hardware/video games

0 voters

Regards,
Jan for the SeisComP development team

4 Likes
3

Hi Jan,

It is great to see such promising plans, I look forward trying the next version, and sorry to the unnecessary PR lately, now I understand my mistake, I’ll redo them in the right place.

I cannot remember version numbers… Hopefully, there will be version names!

Good luck and take care.

4

I really like how the new software repository has been re-arranged. Neat!

Thanks for all the great work!

Luca

5

Congratulations for the 10 years & all the best for the 10 next! Thanks a lot!!

6

Thanks to all the teams involved in and
for this news.
It is very much appreciated for all your efforts in maintaining and making this software always evolving.

Cheers
jerome

7

I am happy to see that many forum users have participated in the poll hoping for even more to join.

Cheers,
Dirk

8

Congratulations for the 10 years , and thanks at all seiscomp´s teams and for the support and continuous efforts.

Best regards,
José Areosa Pena

9

Hello,

This brand new version looks like. I unfortunately miss time to test it right now, but it’s on the todo list.

I read in the changelog that the spread server was changed by an homemade system.
This brings me a few questions.

  • Does the old code that were using the old messaging system using seiscomp3 core libraries will need to be modified ?
    Or is the new messaging handled by the same upper level seiscomp3 libraries ?

  • Will a brand new compiled system be able to connect an old system using spread ?
    I’m thinking, for example, about scimex or scimport for message exchanges, but also at scolv.

I will try to find some time to do some tests anyway.

Regards.

Jean-Marie.

10

Only slightly. Of course the same abstraction level is available as with SC3 but that API has changed a little bit.

Actually I have been waiting for this request from you :wink:

Once reason why we have removed Spread is because of its license which is incompatible with GPL. So we cannot include it into SC. A plugin may be feasible which allows to connect from SC to SC3 and provided externally as addon but that is not yet written. Another (more easy with respect to licenses) way is to implement a plugin for SC3 which allows to connect to the new messaging system. Both would have to be developed.

Regards,
Jan

11

Thanks for all efforts for all the years and for this good news. Great job!

Just wanted to try new SeisComP on new Ubuntu 20.04. First some mysql syntax slightly changed for creating users, so it wasn’t be possible to upload database with table schema from scconfig. But I still successfully loaded it from command line utility.

Also I would mention initial loading of xml inventories of all the stationary networks and temporary networks took more than 12 hours.

alex@alex-Standard-PC-i440FX-PIIX-1996 : ~/.seiscomp/log $ seiscomp install-deps
error: command ‘install-deps’ failed: can only concatenate str (not “bytes”) to str
alex@alex-Standard-PC-i440FX-PIIX-1996 : ~/.seiscomp/log $

There are suspicious messages in scmaster.log:
2020/04/26 18:59:11 [error/MYSQL] execute(“insert into Amplitude(_oid,_parent_oid,amplitude_confidenceLevel,amplitude_lowerUncertainty,amplitude_pdf_used,amplitude_uncertainty,amplitude_upperUncertainty,amplitude_used,amplitude_value,creationInfo_agencyID,creationInfo_agencyURI,creationInfo_author,creationInfo_authorURI,creationInfo_creationTime,creationInfo_creationTime_ms,creationInfo_modificationTime,creationInfo_used,creationInfo_version,evaluationMode,filterID,magnitudeHint,methodID,period_used,pickID,scalingTime_used,snr,timeWindow_begin,timeWindow_end,timeWindow_reference,timeWindow_reference_ms,timeWindow_used,type,unit,waveformID_channelCode,waveformID_locationCode,waveformID_networkCode,waveformID_resourceURI,waveformID_stationCode,waveformID_used) values (269783,1,NULL,NULL,‘0’,NULL,NULL,‘1’,-nan,‘ASBtest’,’’,‘scautopick@alex-Standard-PC-i440FX-PIIX-1996’,’’,‘2020-04-26 11:59:11’,181053,NULL,‘1’,’’,NULL,’’,’’,’’,‘0’,‘20200426.115730.89-II.JTS.00.BHZ’,‘0’,4.639761195,-5.175,59.825,‘2020-04-26 11:57:31’,69538,‘1’,‘mB’,’’,‘BHZ’,‘00’,‘II’,’’,‘JTS’,‘1’)”) = 1054 (Unknown column ‘nan’ in ‘field list’)
2020/04/26 18:59:11 [error/DatabaseArchive] writing object with type ‘Amplitude’ failed
2020/04/26 18:59:11 [warning/dbstore] Error handling message from scautopick to AMPLITUDE
2020/04/26 19:01:15 [error/MYSQL] execute(“insert into Amplitude(_oid,_parent_oid,amplitude_confidenceLevel,amplitude_lowerUncertainty,amplitude_pdf_used,amplitude_uncertainty,amplitude_upperUncertainty,amplitude_used,amplitude_value,creationInfo_agencyID,creationInfo_agencyURI,creationInfo_author,creationInfo_authorURI,creationInfo_creationTime,creationInfo_creationTime_ms,creationInfo_modificationTime,creationInfo_used,creationInfo_version,evaluationMode,filterID,magnitudeHint,methodID,period_used,pickID,scalingTime_used,snr,timeWindow_begin,timeWindow_end,timeWindow_reference,timeWindow_reference_ms,timeWindow_used,type,unit,waveformID_channelCode,waveformID_locationCode,waveformID_networkCode,waveformID_resourceURI,waveformID_stationCode,waveformID_used) values (269819,1,NULL,NULL,‘0’,NULL,NULL,‘1’,-nan,‘ASBtest’,’’,‘scautopick@alex-Standard-PC-i440FX-PIIX-1996’,’’,‘2020-04-26 12:01:15’,121453,NULL,‘1’,’’,NULL,’’,’’,’’,‘0’,‘20200426.115730.89-II.JTS.00.BHZ’,‘0’,6.279993394,-5.575,149.425,‘2020-04-26 11:57:31’,469538,‘1’,‘MLv’,’’,‘BHZ’,‘00’,‘II’,’’,‘JTS’,‘1’)”) = 1054 (Unknown column ‘nan’ in ‘field list’)
2020/04/26 19:01:15 [error/DatabaseArchive] writing object with type ‘Amplitude’ failed
2020/04/26 19:01:15 [warning/dbstore] Error handling message from scautopick to AMPLITUDE
2020/04/26 19:32:28 [error/MYSQL] execute(“insert into Amplitude(_oid,_parent_oid,amplitude_confidenceLevel,amplitude_lowerUncertainty,amplitude_pdf_used,amplitude_uncertainty,amplitude_upperUncertainty,amplitude_used,amplitude_value,creationInfo_agencyID,creationInfo_agencyURI,creationInfo_author,creationInfo_authorURI,creationInfo_creationTime,creationInfo_creationTime_ms,creationInfo_modificationTime,creationInfo_used,creationInfo_version,evaluationMode,filterID,magnitudeHint,methodID,period_used,pickID,scalingTime_used,snr,timeWindow_begin,timeWindow_end,timeWindow_reference,timeWindow_reference_ms,timeWindow_used,type,unit,waveformID_channelCode,waveformID_locationCode,waveformID_networkCode,waveformID_resourceURI,waveformID_stationCode,waveformID_used) values (270257,1,NULL,NULL,‘0’,NULL,NULL,‘1’,inf,‘ASBtest’,’’,‘scautopick@alex-Standard-PC-i440FX-PIIX-1996’,’’,‘2020-04-26 12:32:28’,117388,NULL,‘1’,’’,NULL,’’,’’,’’,‘0’,‘20200426.123205.84-II.IBFO.00.BHZ’,‘0’,NULL,-0,10,‘2020-04-26 12:32:06’,643300,‘1’,‘snr’,’’,‘BHZ’,‘00’,‘II’,’’,‘IBFO’,‘1’)”) = 1054 (Unknown column ‘inf’ in ‘field list’)
2020/04/26 19:32:28 [error/DatabaseArchive] writing object with type ‘Amplitude’ failed
2020/04/26 19:32:28 [warning/dbstore] Error handling message from scautopick to AMPLITUDE

From time to time scolv freezes at start, after splash disappears, no any other windows opened, log:
09:38:19 [debug] Adding plugin path: .
09:38:19 [debug] Adding plugin path: /home/alex/.seiscomp/plugins
09:38:19 [debug] Adding plugin path: /home/alex/seiscomp/lib/plugins
09:38:19 [debug] Adding plugin path: /home/alex/seiscomp/lib
09:38:19 [debug] Adding plugin path: /home/alex/seiscomp/share/plugins
09:38:19 [debug] Adding plugin path: /home/alex/seiscomp/share/plugins/scolv
09:38:19 [debug] Trying to open plugin at /home/alex/seiscomp/share/plugins/dbmysql.so
09:38:19 [info] Plugin dbmysql registered
09:38:19 [debug] Trying to open plugin at /home/alex/seiscomp/share/plugins/mlh.so
09:38:19 [info] Plugin mlh registered
09:38:19 [info]
Plugins:

[1]
description: MySQL database driver
author: GFZ Potsdam seiscomp-devel@gfz-potsdam.de
version: 0.9.2
API: 14.0.0
[2]
description: MLh magnitude method (max of both horizontal compontents)
author: gempa GmbH, modified by Stefan Heimers at the SED
version: 0.0.8
API: 14.0.0

09:38:19 [info] Connect to messaging
09:38:19 [debug] Trying to connect to scolv@localhost/production with primary group = LOCATION
09:38:19 [info] scmp://localhost/production
09:38:19 [debug] Attempt to connect to localhost:18180/production
09:38:19 [debug] Registered clientname: scolv
09:38:19 [info] Outgoing messages are encoded to match schema version 0.11
09:38:19 [info] Want to flush 0 backlog messages
09:38:19 [info] Connect to database
09:38:19 [info] Received database service parameters
09:38:19 [info] Trying to connect to mysql database
09:38:19 [debug] Connected to sysop:******@localhost:3306/seiscomp (Localhost via UNIX socket)
09:38:19 [debug] Found database version v0.11
09:38:19 [info] Connected successfully
09:38:19 [info] Loading complete inventory
09:38:44 [info] Finished loading complete inventory
09:38:44 [info] Loading configuration module
09:38:45 [info] Finished loading configuration module
09:42:19 [debug] Received ack, 19 remaining messages in buffer
09:42:31 [debug] Received ack, 19 remaining messages in buffer
09:42:43 [debug] Received ack, 19 remaining messages in buffer
09:42:55 [debug] Received ack, 19 remaining messages in buffer
09:43:07 [debug] Received ack, 19 remaining messages in buffer
09:43:19 [debug] Received ack, 19 remaining messages in buffer
09:43:31 [debug] Received ack, 19 remaining messages in buffer
09:43:43 [debug] Received ack, 19 remaining messages in buffer
09:43:55 [debug] Received ack, 19 remaining messages in buffer

Regards,
Alex

12

Thank you for the feedback Alex. The amplitude issue is a known issue for some time now. It also appears from time to time on an SC3 installation and I haven’t found the reason for that. It would be very helpful if you were able to do an offline playback with scautopick for a station in question and the time span around the error (around the time of the amplitude measurement). If the value “inf” or “nan” also shows up in the resulting XML then we have a case to reproduce this.

Regarding the scolv freeze, this is indeed not desired. I think that scolv buffers to many outgoing messages and does not receive acknowledgements in time. I have to check that. Maybe the fixes required to support OSX introduced this behaviour. I am going to check that.

Thanks again,
Jan

13

Btw, how did you start scolv? In offline mode? My debug startup sequence looks like this:

...
13:56:38 [info] Loading configuration module
13:56:40 [info] Finished loading configuration module
13:56:40 [info] Found cities.xml and read 71653 entries
13:56:40 [debug] Settings up connection state in settings dialog
13:56:40 [debug] Joining group: CONFIG in settings dialog
13:56:40 [debug] Subscribed to group CONFIG
13:56:40 [debug] Joining group: EVENT in settings dialog
13:56:40 [debug] Subscribed to group EVENT
13:56:40 [debug] Joining group: FOCMECH in settings dialog
13:56:40 [debug] Subscribed to group FOCMECH
13:56:40 [debug] Joining group: GUI in settings dialog
13:56:40 [debug] Subscribed to group GUI
13:56:40 [debug] Joining group: LOCATION in settings dialog
13:56:40 [debug] Subscribed to group LOCATION
13:56:40 [debug] Joining group: MAGNITUDE in settings dialog
13:56:40 [debug] Subscribed to group MAGNITUDE
13:56:40 [debug] Joining group: PICK in settings dialog
13:56:41 [debug] Subscribed to group PICK
...
13:56:45 [info] starting message thread
13:56:46 [debug] Received ack, 0 remaining messages in buffer
13:56:57 [debug] Received ack, 0 remaining messages in buffer

I am missing “starting message thread” in your case.

14

No, not offline, just type “scolv --debug” in commandline . Will check that again.

15

First, --inventory-db doesn’t work?!

alex@alex-Standard-PC-i440FX-PIIX-1996:~$ scautopick --offline --inventory-db inv_II.JTS.xml --playback -I II.JTS.00.BHZ.D.2020.117 --debug
20:56:41 [debug] Adding plugin path: .
20:56:41 [debug] Adding plugin path: /home/alex/.seiscomp/plugins
20:56:41 [debug] Adding plugin path: /home/alex/seiscomp/lib/plugins
20:56:41 [debug] Adding plugin path: /home/alex/seiscomp/lib
20:56:41 [debug] Adding plugin path: /home/alex/seiscomp/share/plugins
20:56:41 [debug] Adding plugin path: /home/alex/seiscomp/share/plugins/scautopick
20:56:41 [debug] Trying to open plugin at /home/alex/seiscomp/share/plugins/dbmysql.so
20:56:41 [info] Plugin dbmysql registered
20:56:41 [debug] Trying to open plugin at /home/alex/seiscomp/share/plugins/mlh.so
20:56:41 [info] Plugin mlh registered
20:56:41 [info]
Plugins:

[1]
description: MySQL database driver
author: GFZ Potsdam seiscomp-devel@gfz-potsdam.de
version: 0.9.2
API: 14.0.0
[2]
description: MLh magnitude method (max of both horizontal compontents)
author: gempa GmbH, modified by Stefan Heimers at the SED
version: 0.0.8
API: 14.0.0

20:56:41 [info] Connect to database
20:56:41 [error] Fetching database parameters failed, no messaging connection
20:56:41 [debug] Leaving ::done
alex@alex-Standard-PC-i440FX-PIIX-1996:~$

Am I doing something wrong?

Than loading from db works
alex@alex-Standard-PC-i440FX-PIIX-1996:~$ scautopick --offline -d sysop:sysop@localhost/seiscomp --playback -I II.JTS.00.BHZ.D.2020.117 --debug

For scmaster log:
2020/04/26 19:01:15 [error/MYSQL] execute(“insert into Amplitude(_oid,_parent_oid,amplitude_confidenceLevel,amplitude_lowerUncertainty,amplitude_pdf_used,amplitude_uncertainty,amplitude_upperUncertainty,amplitude_used,amplitude_value,creationInfo_agencyID,creationInfo_agencyURI,creationInfo_author,creationInfo_authorURI,creationInfo_creationTime,creationInfo_creationTime_ms,creationInfo_modificationTime,creationInfo_used,creationInfo_version,evaluationMode,filterID,magnitudeHint,methodID,period_used,pickID,scalingTime_used,snr,timeWindow_begin,timeWindow_end,timeWindow_reference,timeWindow_reference_ms,timeWindow_used,type,unit,waveformID_channelCode,waveformID_locationCode,waveformID_networkCode,waveformID_resourceURI,waveformID_stationCode,waveformID_used) values (269819,1,NULL,NULL,‘0’,NULL,NULL,‘1’,-nan,‘ASBtest’,’’,‘scautopick@alex-Standard-PC-i440FX-PIIX-1996’,’’,‘2020-04-26 12:01:15’,121453,NULL,‘1’,’’,NULL,’’,’’,’’,‘0’,‘20200426.115730.89-II.JTS.00.BHZ’,‘0’,6.279993394,-5.575,149.425,‘2020-04-26 11:57:31’,469538,‘1’,‘MLv’,’’,‘BHZ’,‘00’,‘II’,’’,‘JTS’,‘1’)”) = 1054 (Unknown column ‘nan’ in ‘field list’)
2020/04/26 19:01:15 [error/DatabaseArchive] writing object with type ‘Amplitude’ failed
2020/04/26 19:01:15 [warning/dbstore] Error handling message from scautopick to AMPLITUDE

I got in XML:

<amplitude publicID="20200426.115730.89-II.JTS.00.BHZ.MLv">
  <type>MLv</type>
  <amplitude>
    <value>-nan</value>
  </amplitude>
  <timeWindow>
    <reference>2020-04-26T11:57:31.469538Z</reference>
    <begin>-5.575</begin>
    <end>149.425</end>
  </timeWindow>
  <snr>6.279993394</snr>
  <pickID>20200426.115730.89-II.JTS.00.BHZ</pickID>

Looks like that we are searching for.

16

Not without providing the config via command line as well. You can dump it with:

scxmldump -C -d localhost > config.xml

and then run scautopick with:

scautopick --config-db config.xml --inventory-db inv.xml

scautopick needs the bindings for the channel selection.

Great. Can you provide me with the files to reproduce it?

Regards,
Jan

17

sent by email

18

Going to check it in a few days after holidays.

19

Thank you. I could reproduce it and I have fixed it hopefully. That error is also present in SeisComP3 and has been fixed there as well.

20

Out of curiosity, could you please point to the patch on github?