Installing COMET

Table of Contents

1. Introduction
2. Installing the COMET server
3. Help! The server isn’t responding - what do I do?
4. Installing the COMET IME
5. Using COMET IME without installing your own server
6. Installing the COMET reports
7. How to report problems with COMET

---

Introduction

The following instructions are provided to help users that want to install their own COMET server, for example for tool evaluation, experiments in digital engineering, or educational purposes at universities, or other concurrent design facilities. It is based on our own experiences deploying COMET operationally in the ESA Concurrent Design Facility, and provided here as-is.

The information is quite detailed and might look daunting at first, but in fact it is not a lot of work for a seasoned system administrator or experienced Linux user, you could have everything up and running in a just a few hours - or less even! If you are familiar with docker, and deploying docker containers, then it should be straightforward.

COMET software is open source and can be used free of charge. Take note that COMET is not ESA owned or provided software, please check the license details before use. ESA does not provide technical support or fix bugs. Commercial support can be obtained from STARION, through their Enterprise license scheme.

It is recommended to first install the COMET IME (client), and familiarise yourselves a bit with the tool using the STARION supplied sandbox server environment that is accessible on the Internet. Once you get the hang of it, deploy your own server, and connect to that from the IME.

CAVEAT

These instructions have been tested with COMET server version 7.2.1 and COMET IME version 9.6.

For the new COMET server version 8, there currently is no docker compose procedure available, however, the process is likewise to what is described here.

Installing the COMET server

The prerequisites to installing the COMET server are as follows:

• You can either use a physical machine or a virtual machine in some cloud infrastructure or your own choosing.
• You can also install it locally on your own desktop machine (or laptop even), if and only if you have sufficient priviliges, so either sudo on Linux or local Administrator on Windows.
• You should have admin priviliges on the server, and it should be visible / accessible in the network where you want to deploy the COMET web services. Typically, you will enable the ports for SSH (22) and HTTPS (443) for network traffic and remote management of that machine, but this may differ in your company set-up.
• If you don’t have access yourselves, liaise with your local system administrator and/or network administrator, so he can make the modifications on your behalf, using this information.
• Any basic installation of a main stream Linux distribution will do, so Debian, Ubuntu or RedHat are all fine; use LTS versions for security patches, stability and support; bleeding edge Linux versions (like ArchLinux) might work : good luck with that!
• Want to deploy on Windows 10/11 or Windows server machines? If you have a license for Docker Desktop, then you can install it pretty much as per below. But the Docker Engine route on Windows is not recommended. Idem for installation via WSL, yes it can be done, but it is not recommended. Keep it simple and easy: use a standard Linux distro!
• For the sizing of the machine, at least 1 CPU, 4GB of free RAM, and 50 GB free disk space is recommended (100GB, if also including the OS installation on a virgin machine). Access to SSD disks is recommended for better database performance, in stead of Tier-2 or Tier-3 storage solutions.
• Be aware that antivirus or malware scanners (like amp) may significantly hamper performance, for example if the exposed COMET port is actively monitored; allow for more RAM to compensate, or disable these scanners on this port only.
• COMET can also be added to existing server instances, but then make sure the machine is properly sized! COMET does not add a high average loads, but is more sensitive to sudden peak loads (CPU and network), due to the user interactions, that require short response times.
• Configure the system in compliance to your local secure management and operations policies, and test network access from the client machines in your network in advance (e.g. using curl or something similar).

It is strongly recommended (no, needed!) to run COMET behind a reverse proxy, with a valid SSL certificate, to provide access to COMET over a secure HTTPS connection. The reverse proxy does not necessarily need to run on the same physical machine. If you already have another machine that provides this functionality, then of course you need to modify the configuration on that machine (or ask some sysadmin to do it for you). You can use apache, traefik or nginx; here we will assume the latter, NGINX installed on the same machine that will deploy COMET.

• Install the latest stable version of NGINX, and obtain an SSL certificate from whatever certificate authority that you (or your organisation) prefer; install that SSL certificate as per standard NGINX documentation, and test it with some simple static html data, from a web browser running on a client machine.
• For NGINX, use the following basic (and secure) SSL configuration (typically in /etc/nginx/nginx.conf):

        server {
                listen       443 ssl http2;
                listen       [::]:443 ssl http2;
                server_name  _;
  
                ssl_certificate "/wherever/yours/is";
                ssl_certificate_key "/wherever/that/one/is";
                ssl_session_timeout 1d;
                ssl_session_cache shared:SSL:10m;
                ssl_session_tickets off;
  
                # enforce strong cyphers only and strict CORS
                ssl_protocols TLSv1.2 TLSv1.3;
                ssl_ciphers "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH+aRSA+RC4 EECDH EDH+aRSA HIGH !RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS";
                ssl_prefer_server_ciphers on;
                add_header Strict-Transport-Security max-age=31536000;
  
                location / {
                    # configure to point to some static HTML for testing
                    # see later on how to reconfigure to point to COMET
                }
        }
  

• If you already have an operational reverse proxy that you want to modify, then install a firewall on the machine that will deploy COMET (e.g. ufw on Ubuntu, or something similar), and open a TCP port of your own choosing (e.g. port 5000) and allow access only from the remote reverse proxy IP address to that port. In that way, the COMET server cannot be reached except through the reverse proxy running on the other machine. Please test this set-up before you continue; start some standard Linux service (like telnet, or a web server) on the target server/port and verify that you can access it from the machine that runs the reverse proxy, on that particular exposed port!

The next steps are to:

• login to the server system, using some sudo-er enabled account; do not install COMET as root!
• Install docker engine and docker compose, you can find all instructions on https://docs.docker.com. Note that modern versions of docker have compose integrated - so no need any more to install that separately.
• Do not forget to perform the docker post-installation steps (creating a docker group and adding the user to that group - to avoid having to run docker as sudo user all the time).
• Test the docker installation by running the docker run hello-world example.
• Ensure that docker is installed to automatically restart its own services on system reboot, using systemctl. Test this by actually rebooting the server, logging in again, and try running the hello-world example again.
• Test the docker compose installation by running the docker compose version command; for older / existing docker installations, it might still be the legacy docker-compose version command (note the dash). It is however strongly recommended to use a recent docker (compose) version!
• Ensure that the git client software is installed and properly configured for the current user.

All the stuff above, is basic (but secure) server configuration, we are now finally ready to install the COMET server on this machine:

• Perform the following command: git clone https://github.com/STARIONGROUP/COMET-Docker-Compose.git.
• Enter the COMET-Docker-Compose directory, and create a new git branch to store local configuration changes, e.g. run git checkout myconfig.
• Modify the compose.sh script by making it executable, using chmod +x compose.sh; edit the file, possibly you need to change all docker-compose commands by docker compose in the script, if you have the latest version of docker installed. You can also set the port number for the COMET web-server here, default is 5000. If you want to use another port, then the only change you need to make is here. Likewise, if you already have a postgresql server running on the default 5432 port, then modify the setting in this script, to avoid a clash with other services. If you want to find out if there are conflicting services that are already using these ports, you can use netstat -tulpn | grep LISTEN
• Modify the docker-compose.yml file; note there are two ports lines in the script! For the cdp4_db configuration, modify the line below ports into - '127.0.0.1:${DB_HOSTPORT}:5432'; this will limit the exposure of the ${DB_HOSTPORT} only to the localhost. Likewise, for the cdp4_webservices configuration and running NGINX on the same machine, then modify the line below ports into - '127.0.0.1:${WEBSERVICES_HOSTPORT}:5000'. If you run the reverse proxy on another machine, then leave the line as-is. You can also use localhost in stead of 127.0.0.1.
• Save the file and now run sh ./compose.sh up and keep fingers crossed; you should see docker pulling all containers from the public docker registry and then starting the containers. This may take a few minutes, be patient (in particular if your internet connection to docker hub is slow or throttled).
• Caveat: docker hub may require login (using the docker login command, prior to executing the compose.sh script) in case your organisation is making many concurrent pull requests from the same ip-address! In that case, please ensure you create a docker hub account to allow this.
• Once compose.sh is finished, you can check the status of the docker containers, using docker ps -a; this should show that the two COMET services are indeed up and running!
• Check the response of the COMET web services by performing curl localhost:5000, which should give you a html file response (indicating the server is indeed up)
• It is good practice to save this new configuration, so use git add and git commit to put it under configuration control. For all future changes, follow the same process. Do not push to remote - it is just to capture your local configuration history!

The final step is to ensure the NGINX reverse proxy is properly configured, and points to the COMET server

• Modify the NGINX server configuration for the SSL connection, by adding or modifying:

                location / {
                        proxy_pass http://127.0.0.1:5000;
                        proxy_read_timeout 120;
                }
  

• If you already have other services running, you can use any arbitrary root, but you need to use a rewrite rule, to map the URI to the downstream COMET web service:

                location /comet {
                        rewrite ^/comet/(.*) /$1 break;
                        proxy_pass http://localhost:5000;
                        proxy_read_timeout 120;
                }
  

• If the NGINX server is not on the same physical machine, then replace 127.0.0.1 by the FQDN or ip address of the server where COMET is running.
• Test the new nginx configuration (sudo nginx -t) and the restart the NGINX services (sudo systemctl restart nginx); you can verify the nginx status again with sudo systemctl status nginx.
• Let’s assume that the FQDN of your server (for which you obtained the SSL certificate) is comet.mydomain.org, now you can test the secure connection to the COMET server, by trying curl https://comet.mydomain.org and that should again return the html page like before (but now over https). Try it both on the server and client machines. If you are not using the server root, like shown in the second location example above, then use curl https://comet.mydomain.org/comet/ in stead (similiar approach for the other examples that follow below).
• Finally, open a normal web-browser on any client machine in your network and access the same URL, this should present again the COMET server splash page - check the validity of the SSL connection in your browser!
• If you are using Chrome, then proceed to https://comet.mydomain.org/SiteDirectory and you will get a security challenge pop-up, asking for a username and password; use the default credentials (admin/pass) and the server should return with some JSON data.
• You can do a similar test with curl, like this: curl https://comet.mydomain.org/SiteDirectory -u "admin:pass", which should also return the JSON data.
• It is recommended to close your browser and reboot the server, and try the last few steps again, to ensure that the server comes back up after a full power cycle. Just to be sure, either clear your browser cache or use a new incognito tab, to ensure you are not looking at cached data from your previous test…
• The last step is obviously to change the default credentials for the admin account for which you can use the COMET IME (see below).
• Last but not least, if your organisation supports it, run an independant security audit on the machine, to ensure that everything is a-ok, and you did not forgot anything.

And one more thing or two…

• Ensure that the server or VM is in some regular back-up policy.
• Enable automatic installation of security patches for the operating system; otherwise perform regular checks on the server to ensure this is done manually.
• Hook this server up in your corporate monitoring infrastructure, so you can keep an eye on performance and availability.
• It is recommended to create a daily dump of the postgresql database, and store it on another file server - for safe keeping. A simple crontab can do the job, for example to run at 3AM every weekday. Here is a basic example shell script, modify as you see fit!

#! /usr/bin/bash

# some local environment variables for directory and file names

# account name of the Linux user that runs the back-up script
# caveat: this Linux user must be part of the 'docker' group!
USER=

# the name of the Linux server that runs COMET
SERVER=

# the generated names of the back-up and log files
FNAME=`date +\%Y\%m\%d`-dump.sql
DNAME=/home/$USER/backup
LNAME=/home/$USER/logs/$SERVER.`date +\%Y\%m\%d`.log
# caveat: create the backup and logs directory before running this script the first time

# print some diagnostics to the log file
echo -n "*** Started backup at "
date

echo "Check docker container status before backup"
docker ps -a

echo "Disk status before backup"
df

# first get the server logs from the last day
echo "Retrieving COMET server log"
docker cp comet-docker-compose_cdp4_webservices_1:/app/logs/CDP4Server.log.0 $LNAME
docker exec comet-docker-compose_cdp4_webservices_1 rm -f /app/logs/CDP4Server.log.0
ls -alh $LNAME

# now start the actual back-up
echo "Commence backup"

# stop the COMET WSP to ensure no-one accesses the postgres database
echo "stop COMET WSP docker container"
docker stop comet-docker-compose_cdp4_webservices_1

# create an in-situ database dump in the postgres docker container
echo "dump the database"
docker exec cdp4-database-community-edition pg_dumpall --clean --quote-all-identifiers -f /tmp/wspdump.sql -U postgres --no-password

# move the backup to the local file store
echo "retrieve the backup file from the docker container"
docker cp cdp4-database-community-edition:/tmp/wspdump.sql $DNAME/$FNAME
docker exec cdp4-database-community-edition rm -f /tmp/wspdump.sql

# compress the backup we made (optional caveat: ... and encrypt it)
echo "compress the backup file"
gzip $DNAME/$FNAME

# identify the backup we made
echo "identify the backup file"
sha512sum $DNAME/$FNAME.gz

# make sure only the most recent seven are kept locally
find $DNAME -type f -mtime +7 -name '*.gz' -delete

# move to the backup directory
cd $DNAME

echo "contents of the backup directory"
ls -al *.gz

# push the resulting file to the remote archive
echo "copy backup file to CDF file store"
# caveat: use rcp, rsync or smbclient to copy $FNAME.gz to another file store or back-up server

# restart the COMET WSP services
echo "Restart the COMET WSP docker container"
docker start comet-docker-compose_cdp4_webservices_1

# add docker container status to the log file
echo "Docker container status"
docker ps -a

# print some final statistics to the log file
echo -n "*** Finished backup at "
date
echo ""

Help! The server isn’t responding - what do I do?

Here are a few tips to quickly diagnose what is going on in case of complaints from users:

• If the complaints come from a single user, check the local network connection (in particular if it is via WiFi), and access credentials. When local operating system updates are pushed to client machines, this sometimes leads to system instability. Use the classic Windows repair option: reboot, wait, wait some more, cross fingers and try again!
• First, test the network connection to the server, either using ping or curl. If the server cannot be reached, check your network connection and network settings.
• If the network seems to respond properly with ping, then open a normal web-browser and a new incognito window, and go to the top-level URL of your COMET server, e.g. https://comet.mydomain.org and you should see the COMET splash page appear.
• If the splash page appears without problems, then open a new incognito window, and access https://comet.mydomain.org/SiteDirectory. Use the credentials of the complaining user to gain access, you should see a JSON response. If the JSON response seems fine, then the problem is 100% on the client machine of the user for sure (the server is fine) - which may require reinstalling the COMET IME client, as a last resort.
• If the JSON response is broken, or not appearing at all, then try again with the web browser approach, but using another account or the admin account. If other account works, then you probably have to reset the credentials of the complaining user, and try again. If you also fail to get a response under the admin user, only then it is time to check the server itself.
• If you have a reverse proxy that is not deployed on the same machine as the COMET server, then go and check if there are no issues on that machine, and check also the performance, the logs and check the configuration of the reverse proxy service.
• Idem for checking with your network administrator, if any recent changes may have caused interference in the networking connections.
• Login to the server onto which COMET is deployed, using the account details used during installation.
• Do a quick sanity check on the server itself (available disk space using df, the overall status of the machine with top and systemctl status, and check the system and kernel logs for anything out of the ordinary), if there are issues at OS level, fix them and perform a system reboot, and proceed from the top again.
• Double check other services hosted on this machine, such as firewalls, virus and malware scanners, and of course the reverse proxy server, if that is deployed on the same server. Make sure you check the validity of the SSL certificate, are you sure it hasn’t expired yesterday?
• If the OS level and networking sanity check seems fine, then check the status of the docker containers using docker ps -a and check the status of the two cdp4 containers, they should be up and running. If they are not running, you need to figure out why they are stopped. You can restart them using the compose.sh script, or by using docker restart. By default, the containers are set to auto restart, so it is likely they might stop again, so check the container logs for any errors, using docker logs. If the container boots up again and stays up, then you can use the docker exec -it comet-docker-compose_cdp4_webservices_1 bash command to login the container, and check the CDP4 web services log. For example, you can use tail -f /app/logs/CDP4Server.log and then access the system from the network to see what happens during interaction with the server. Remember that you can trigger access to the COMET server using curl https://comet.mydomain.org/SiteDirectory -u "admin/pass" (of course using the right credentials). Check the server log for the ERROR keyword - if it appears, it may give a hint to what is wrong.
• After diagnosis and fixes, it might be a good idea to simply perform a full server reboot, to ensure no residuals are left in caches that might cause further instabilities. After reboot, perform some of the sanity checks shown above to cross check.
• If all this does not allow you to diagnose the problem, then it is probably time to contact STARION or submit a bug report.

Installing the COMET IME

The COMET IME only runs on Windows 10 or Windows 11. Do not attempt to run it with Wine on Linux or Rosetta on Mac - resistance is futile! Work is under way to create a full web-based user interface, that can be run from any browser. The remainder of this text, is on the installation of the IME on a Windows desktop.

• You must have local administrator rights to install the IME; if you don’t have this, then liaise with your system administrator to help you get approval and install it.
• COMET IME also provides plug-ins for Excel; these only work with a local installation of Excel, it does not work with cloud hosted Excel versions!
• If you have old versions of either OCDT or COMET installed: remove these first and reboot your machine!
• You can find the MSI installers for COMET IME here: https://github.com/STARIONGROUP/COMET-IME-Community-Edition/releases/latest
• If you have 64-bit Excel, then use the 64-bit installer!
• If you have 32-bit Excel, then use the 32-bit installer!
• Caveat: 64-bit Excel, with 32-bit COMET IME: does not work!
• Caveat: 32-bit Excel, with 64-bit COMET IME: does not work!
• Download and install as per above - suggest to do a full install, including all plug-ins
• If you don’t have Excel installed locally, use the 64-bit version and install without the Excel plug-ins.
• Start the IME, and press the “Connect” symbol.
• Push the button next to the “Source address” field and create a new entry for your COMET server URL.
• Login to COMET with your credentials. First time access: the default is admin/pass. After successful login, immediately go to the Directory tab, then to the Users icon, and select the Administrator account and modify the password! It is recommended to store that information into some keepass file that is shared with your system administrator. If you loose the master password, it cannot be recovered!
• The official CDP4-COMET documentation (PDF), provided by STARION

Using COMET IME without installing your own server

STARION provides access to publically hosted COMET server that acts as a testing sandbox. The contents of this database is cleared every day, so do NOT use it to store any data that should have lasting value, or is proprietary or confidential. It is a public server, to which anyone can have access concurrently. The URI for that server is https://cdp4services-public.cdp4.org/ with credentials admin/pass.

Installing the COMET reports

The templates used in ESA studies for customised COMET mass and power reports can be found at https://github.com/STARIONGROUP/COMET-Reports/releases/latest

How to report problems with COMET

To report ESA CDF internal ICT issues (CITRIX or COMET), please contact the CDF AV/IT service technician.

For all external users of COMET:

• For the COMET server, please use https://github.com/STARIONGROUP/COMET-WebServices-Community-Edition/issues
• For the COMET IME, please use https://github.com/STARIONGROUP/COMET-IME-Community-Edition/issues

Caveat: to submit issues on github, you need to have your own github account.

NOTE

ESA CDF does not provide support for fixing bugs in COMET.