How to BVQ Docker Update
Dominik Karch
Thomas Sikora
Owned by Dominik Karch
About this task
BVQ has to be updated
Applies to
BVQ Docker
For an Offline-Update, place the files in a specific folder on the Docker host. An update is only possible to a specific version, depending on the version of BVQ currently running. This is also true for the running MongoDB instance; a higher BVQ version may only work with a corresponding or higher MongoDB version. The process of updating MongoDB is similar to the "standard" update process for the Server, Scanner, and Grafana. docker-bvq-grafana-2023.HX.X.tar.gz docker-bvq-scanner-2023.HX.X.tar.gz docker-bvq-server-2023.HX.X.tar.gz
Procedure
- Connect the the host
-Switch to root user
expand
su - |
|---|
2. List running processes
expand
docker ps |
|---|
[bvq@bvqlinux2 ~]$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e03ac5d96220 registry.labwi.sva.de/bvq/bvq-grafana-dev:2022.H2.x "/run.sh" 12 days ago Up 12 days 3000/tcp bvq-grafana
fa9c8584c3ec registry.labwi.sva.de/bvq/bvq-server-dev:2022.H2.x "sh -c 'java ${JAVA_…" 12 days ago Up 12 days 0.0.0.0:80->8080/tcp, :::80->8080/tcp bvq-server
ff6778dde20e registry.labwi.sva.de/bvq/bvq-scanner-dev:2022.H2.x "sh -c 'java ${JAVA_…" 12 days ago Up 12 days 0.0.0.0:2222->2222/tcp, :::2222->2222/tcp bvq-scanner
dde7576a483f mongo:5.0 "docker-entrypoint.s…" 6 weeks ago Up 2 weeks 0.0.0.0:27017->27017/tcp, :::27017->27017/tcp bvq-mongodb
5d647c102f8d traefik:latest "/entrypoint.sh trae…" 6 weeks ago Up 2 weeks 0.0.0.0:443->443/tcp, :::443->443/tcp, 0.0.0.0:3000->3000/tcp, :::3000->3000/tcp, 80/tcp, 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp bvq-traefik
019eecba538d registry.labwi.sva.de/bvq/bvq-server-dev:BVQ-16763-itsm_service_now "sh -c 'java ${JAVA_…" 7 weeks ago Up 2 weeks 0.0.0.0:8081->8080/tcp, :::8081->8080/tcp bvq-server-BVQ-16763-itsm_service_now
5c13aa3c40cd portainer/portainer-ce:latest "/portainer" 3 months ago Up 2 weeks 0.0.0.0:8000->8000/tcp, :::8000->8000/tcp, 0.0.0.0:9000->9000/tcp, :::9000->9000/tcp, 9443/tcp portainer
aca7c41b0927 mongo:4.4 "docker-entrypoint.s…" 4 months ago Up 2 weeks 0.0.0.0:27018->27017/tcp, :::27018->27017/tcp bvq-mongodb-BVQ-16763-itsm_service_now
62b921699641 registry.labwi.sva.de/bvq/bvq-scanner-dev:BVQ-16763-itsm_service_now "sh -c 'java ${JAVA_…" 4 months ago Up 2 weeks 0.0.0.0:2223->2222/tcp, :::2223->2222/tcp bvq-scanner-BVQ-16763-itsm_service_now
57498d399f14 docker.elastic.co/beats/filebeat:7.12.0 "/usr/bin/tini -- /u…" 18 months ago Up 2 weeks logging_filebeat_1
3. Check the server performance
expand
docker stats |
|---|
[bvq@bvqlinux2 ~]$ docker stats CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS e03ac5d96220 bvq-grafana 0.38% 107.7MiB / 1GiB 10.52% 279MB / 372MB 4.81GB / 50.4MB 29 fa9c8584c3ec bvq-server 28.82% 7.889GiB / 8GiB 98.61% 1.7TB / 332GB 15.4GB / 95.2kB 312 ff6778dde20e bvq-scanner 23.31% 2.945GiB / 3GiB 98.17% 956GB / 95.2GB 39.5GB / 41.7GB 934 dde7576a483f bvq-mongodb 16.69% 3.6GiB / 4GiB 90.00% 428GB / 2.22TB 987GB / 2.94TB 57 5d647c102f8d bvq-traefik 0.02% 49.53MiB / 27.27GiB 0.18% 1.16GB / 1.16GB 3.94GB / 0B 14 019eecba538d bvq-server-BVQ-16763-itsm_service_now 13.78% 3.96GiB / 4GiB 98.99% 825GB / 168GB 2.63GB / 4.1kB 215 5c13aa3c40cd portainer 0.02% 23.15MiB / 27.27GiB 0.08% 29.3kB / 0B 1.3GB / 756MB 11 aca7c41b0927 bvq-mongodb-BVQ-16763-itsm_service_now 4.89% 2.918GiB / 4GiB 72.95% 150GB / 825GB 115GB / 1.11TB 60 62b921699641 bvq-scanner-BVQ-16763-itsm_service_now 12.63% 1.878GiB / 2GiB 93.88% 374GB / 36.1GB 3.72GB / 33.8GB 303 57498d399f14 logging_filebeat_1 1.04% 111.3MiB / 27.27GiB 0.40% 174MB / 7.71GB 3.62GB / 2.49MB 15
4. Move to the folder where the new BVQ files are located and unzip all 3 files
expand...
*the additional "-d" removes the zipped file afterwards
gzip -d docker-bvq-XXXXX-2023.H2.4.tar.gz |
|---|
[bvq@bvqlinux2 tmp]$ ls docker-bvq-grafana-2023.H2.4.tar.gz docker-bvq-scanner-2023.H2.4.tar.gz docker-bvq-server-2023.H2.4.tar.gz [bvq@bvqlinux2 tmp]$ gzip -d docker-bvq-grafana-2023.H2.4.tar.gz [bvq@bvqlinux2 tmp]$ gzip -d docker-bvq-scanner-2023.H2.4.tar.gz [bvq@bvqlinux2 tmp]$ gzip -d docker-bvq-server-2023.H2.4.tar.gz [bvq@bvqlinux2 tmp]$ ls docker-bvq-grafana-2023.H2.4.tar docker-bvq-scanner-2023.H2.4.tar docker-bvq-server-2023.H2.4.tar
5. Load and restore images
expand...
docker load -i docker-bvq-grafana-2023.H2.4.tar |
|---|
[bvq@bvqlinux2 tmp]$ docker load -i docker-bvq-grafana-2023.H2.4.tar [bvq@bvqlinux2 tmp]$ docker load -i docker-bvq-scanner-2023.H2.4.tar [bvq@bvqlinux2 tmp]$ docker load -i docker-bvq-server-2023.H2.4.tar
6. Check the new images
expand...
docker images |
|---|
[bvq@bvqlinux2 bvq-fullstack]$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE registry.labwi.sva.de/bvq/bvq-server-dev 2023.H2.x 637e448b56ff 12 days ago 845MB registry.labwi.sva.de/bvq/bvq-scanner-dev 2023.H2.x 59beba490ec9 12 days ago 408MB registry.labwi.sva.de/bvq/bvq-server-dev <none> 49958f533099 2 weeks ago 845MB registry.labwi.sva.de/bvq/bvq-scanner-dev <none> 423367487d2c 2 weeks ago 408MB registry.labwi.sva.de/bvq/bvq-grafana-dev 2023.H2.x 54a2561cf773 4 weeks ago 301MB
7. Move to the correct BVQ folder
expand
cd .. |
|---|
ls |
|---|
cd ./bvq-data/scripts/bvq-fullstack |
|---|
or check for the correct folder
[bvq@bvqlinux2 ~]$ dir [bvq@bvqlinux2 ~]$ cd .. [bvq@bvqlinux2 home]$ dir bvq test test2 [bvq@bvqlinux2 home]$ cd .. [bvq@bvqlinux2 /]$ dir bin boot bvq-data dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var [bvq@bvqlinux2 /]$ cd bvq-data [bvq@bvqlinux2 bvq-data]$ ls -a . .. bvq-grafana bvq-grafana_vmware_lan bvq-itsm-data bvq-mongodb bvq-server bvq-vmware-lan-data lost+found scripts [bvq@bvqlinux2 bvq-data]$ cd scripts [bvq@bvqlinux2 scripts]$ ls -a . .. bvq-fullstack bvq-itsm logging vmware_lan [bvq@bvqlinux2 scripts]$ cd bvq-fullstack [bvq@bvqlinux2 bvq-fullstack]$ ls -a . .. bvq.crt bvq.crt.old bvq.key bvq.key.old docker-compose.yml dynamic_conf.yml .env grafana LICENSE mongod.conf README.md traefik.yml update_to_latest.sh
8. You do not find the file?
expand
find / -name .env |
|---|
[bvq@bvqlinux2 bvq-fullstack]$ find / -name .env
ind / -name
9. Open the env file
expand
vi .env |
|---|
[bvq@bvqlinux2 bvq-fullstack]$ ls -a . .. bvq.crt bvq.crt.old bvq.key bvq.key.old docker-compose.yml dynamic_conf.yml .env grafana LICENSE mongod.conf README.md traefik.yml update_to_latest.sh [bvq@bvqlinux2 bvq-fullstack]$ vi .env
x# Name of BVQ branch / docker image tag, see bamboo or # https://hub.docker.com/r/svade/bvq-server-dev/tags # https://hub.docker.com/r/svade/bvq-scanner-dev/tags # # Name of the BVQ Stage. There are 3 Stages available in DockerHub # '-dev' -> Development Stage with all Branches # '-beta' -> Beta Stage (aka family & friends release) # no postfix -> Releases (each version with a separate image tag) # Please leave empty if you want to test releases # Remember to set BVQ_BRANCH BVQ_STAGE=-dev BVQ_BRANCH=2022.H2.x # MongoDB version, see https://hub.docker.com/_/mongo?tab=tags DB_VERSION=5.0 # Grafana version, see https://hub.docker.com/r/grafana/grafana/tags # Obsolete when using bvq-grafana image #GRAFANA_VERSION=6.6.2 # User config BVQ_USER=bvq BVQ_PWD=P@ssw0rd # Network config -> exposed container ports, externally reachable DB_PORT=27017 SERVER_PORT=80 SCANNER_PORT_SSH=2222 GRAFANA_PORT=3000 # DB config # DB_HOST must match mongodb container name in docker-compose.yml DB_HOST=bvq-mongodb DB_NAME=bvq # CPU config DB_CPUS=2 GRAFANA_CPUS=2 SCANNER_CPUS=1 SERVER_CPUS=6 # Memory config DB_MEM=4gb GRAFANA_MEM=1gb SCANNER_MEM=3gb SERVER_MEM=8gb #Timezone -> Docker containers run in UTC when not set TZ=Europe/Berlin DOCKER_HOST_VM=BVQLINUX2 INFLUXDB_HOST=10.10.130.57 INFLUXDB_PORT=8086 MANAGEMENT_METRICS_EXPORT_INFLUX_ENABLED=true MANAGEMENT_METRICS_EXPORT_INFLUX_DB=jvm ~ ~ ~ ~ ~ ~ ".env" [readonly] 57L, 1402C
10. Modify the env file
-use "i" to write and change the BVQ_Branch to the new Version 2022.H2.2 → 2023.H2.4
-if you are also updating the MongoDB version, the DB_Version has to be adjusted as well
expand...
x# Name of BVQ branch / docker image tag, see bamboo or # https://hub.docker.com/r/svade/bvq-server-dev/tags # https://hub.docker.com/r/svade/bvq-scanner-dev/tags # # Name of the BVQ Stage. There are 3 Stages available in DockerHub # '-dev' -> Development Stage with all Branches # '-beta' -> Beta Stage (aka family & friends release) # no postfix -> Releases (each version with a separate image tag) # Please leave empty if you want to test releases # Remember to set BVQ_BRANCH BVQ_STAGE=-dev BVQ_BRANCH=2022.H2.x <--------------------- DB_VERSION=5.0 <---------------------
2. Use ESC & ":" and "wq" to save the file
#Timezone -> Docker containers run in UTC when not set TZ=Europe/Berlin DOCKER_HOST_VM=BVQLINUX2 INFLUXDB_HOST=10.10.130.57 INFLUXDB_PORT=8086 MANAGEMENT_METRICS_EXPORT_INFLUX_ENABLED=true MANAGEMENT_METRICS_EXPORT_INFLUX_DB=jvm ~ ~ ~ ~ ~ ~ ~ :wq
11. Recreate server and scanner
expand
docker-compose up -d |
|---|
[bvq@linux2 bvq-fullstack]$ docker-compose up -d Recreating bvq-mongodb ... done Recreating bvq-server ... done Recreating bvq-scanner ... done Recreating bvq-grafana ... done [bvq@linux2 bvq-fullstack]$
12. -if the compose command is not working, keep in mind that the new images have to be named and tagged similar to the old images, you can change both values using the tag command:
expand...
[root@bvqlinux2 bvq-fullstack]# docker-compose up -d Pulling bvq-server (registry.labwi.sva.de/bvq/bvq-server-dev:2023.H2.2)... ERROR: unauthorized: unauthorized to access repository: bvq/bvq-server-dev, action: pull: unauthorized to access repository: bvq/bvq-server-dev, action: pull [root@bvqlinux2 bvq-fullstack]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE svade/bvq-server 2023.H2.2 c776b6b2042c 6 days ago 1.76GB svade/bvq-scanner 2023.H2.2 5db8230feaec 6 days ago 535MB svade/bvq-grafana 2023.H2.2 faa42f30b7ad 6 days ago 297MB mongo 5.0 12ce459b546c 11 days ago 690MB registry.labwi.sva.de/bvq/bvq-server-dev 2023.H2.x e213916d3394 2 weeks ago 1.76GB registry.labwi.sva.de/bvq/bvq-scanner-dev 2023.H2.x 2212514c2a40 2 weeks ago 535MB registry.labwi.sva.de/bvq/bvq-grafana-dev 2023.H2.x 39898a9daf52 2 weeks ago 297MB [root@bvqlinux2 bvq-fullstack]# docker pull svade/bvq-server:2023.H2.2 2023.H2.2: Pulling from svade/bvq-server Digest: sha256:d6d5d200cb27e759d6628353cea1ede4a5a398631a15837147266d84374e5c33 Status: Image is up to date for svade/bvq-server:2023.H2.2 docker.io/svade/bvq-server:2023.H2.2 [root@bvqlinux2 bvq-fullstack]# docker tag svade/bvq-server:2023.H2.2 registry.labwi.sva.de/bvq/bvq-server-dev:2023.H2.2 [root@bvqlinux2 bvq-fullstack]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE svade/bvq-server 2023.H2.2 c776b6b2042c 6 days ago 1.76GB registry.labwi.sva.de/bvq/bvq-server-dev 2023.H2.2 c776b6b2042c 6 days ago 1.76GB svade/bvq-scanner 2023.H2.2 5db8230feaec 7 days ago 535MB svade/bvq-grafana 2023.H2.2 faa42f30b7ad 7 days ago 297MB mongo 5.0 12ce459b546c 11 days ago 690MB registry.labwi.sva.de/bvq/bvq-server-dev 2023.H2.x e213916d3394 2 weeks ago 1.76GB registry.labwi.sva.de/bvq/bvq-scanner-dev 2023.H2.x 2212514c2a40 2 weeks ago 535MB registry.labwi.sva.de/bvq/bvq-grafana-dev 2023.H2.x 39898a9daf52 2 weeks ago 297MB [root@bvqlinux2 bvq-fullstack]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE registry.labwi.sva.de/bvq/bvq-server-dev 2023.H2.2 c776b6b2042c 6 days ago 1.76GB registry.labwi.sva.de/bvq/bvq-scanner-dev 2023.H2.2 5db8230feaec 7 days ago 535MB registry.labwi.sva.de/bvq/bvq-grafana-dev 2023.H2.2 faa42f30b7ad 7 days ago 297MB mongo 5.0 12ce459b546c 11 days ago 690MB registry.labwi.sva.de/bvq/bvq-server-dev 2023.H2.x e213916d3394 2 weeks ago 1.76GB registry.labwi.sva.de/bvq/bvq-scanner-dev 2023.H2.x 2212514c2a40 2 weeks ago 535MB registry.labwi.sva.de/bvq/bvq-grafana-dev 2023.H2.x 39898a9daf52 2 weeks ago 297MB [root@bvqlinux2 bvq-fullstack]# docker-compose up -d bvq-mongodb is up-to-date bvq-traefik is up-to-date Recreating bvq-scanner ... done Recreating bvq-server ... done Recreating bvq-grafana ... done
13. list running processes
expand
docker ps |
|---|
[bvq@bvqlinux2 ~]$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e03ac5d96220 registry.labwi.sva.de/bvq/bvq-grafana-dev:2023.H2.x "/run.sh" 12 days ago Up 12 days 3000/tcp bvq-grafana
fa9c8584c3ec registry.labwi.sva.de/bvq/bvq-server-dev:2023.H2.x "sh -c 'java ${JAVA_…" 12 days ago Up 12 days 0.0.0.0:80->8080/tcp, :::80->8080/tcp bvq-server
ff6778dde20e registry.labwi.sva.de/bvq/bvq-scanner-dev:2023.H2.x "sh -c 'java ${JAVA_…" 12 days ago Up 12 days 0.0.0.0:2222->2222/tcp, :::2222->2222/tcp bvq-scanner
dde7576a483f mongo:5.0 "docker-entrypoint.s…" 6 weeks ago Up 2 weeks 0.0.0.0:27017->27017/tcp, :::27017->27017/tcp bvq-mongodb
5d647c102f8d traefik:latest "/entrypoint.sh trae…" 6 weeks ago Up 2 weeks 0.0.0.0:443->443/tcp, :::443->443/tcp, 0.0.0.0:3000->3000/tcp, :::3000->3000/tcp, 80/tcp, 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp bvq-traefik
019eecba538d registry.labwi.sva.de/bvq/bvq-server-dev:BVQ-16763-itsm_service_now "sh -c 'java ${JAVA_…" 7 weeks ago Up 2 weeks 0.0.0.0:8081->8080/tcp, :::8081->8080/tcp bvq-server-BVQ-16763-itsm_service_now
5c13aa3c40cd portainer/portainer-ce:latest "/portainer" 3 months ago Up 2 weeks 0.0.0.0:8000->8000/tcp, :::8000->8000/tcp, 0.0.0.0:9000->9000/tcp, :::9000->9000/tcp, 9443/tcp portainer
aca7c41b0927 mongo:4.4 "docker-entrypoint.s…" 4 months ago Up 2 weeks 0.0.0.0:27018->27017/tcp, :::27018->27017/tcp bvq-mongodb-BVQ-16763-itsm_service_now
62b921699641 registry.labwi.sva.de/bvq/bvq-scanner-dev:BVQ-16763-itsm_service_now "sh -c 'java ${JAVA_…" 4 months ago Up 2 weeks 0.0.0.0:2223->2222/tcp, :::2223->2222/tcp bvq-scanner-BVQ-16763-itsm_service_now
57498d399f14 docker.elastic.co/beats/filebeat:7.12.0 "/usr/bin/tini -- /u…" 18 months ago Up 2 weeks logging_filebeat_1
14. Check the server performance
expand
docker stats |
|---|
[bvq@bvqlinux2 ~]$ docker stats CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS e03ac5d96220 bvq-grafana 0.38% 107.7MiB / 1GiB 10.52% 279MB / 372MB 4.81GB / 50.4MB 29 fa9c8584c3ec bvq-server 28.82% 7.889GiB / 8GiB 98.61% 1.7TB / 332GB 15.4GB / 95.2kB 312 ff6778dde20e bvq-scanner 23.31% 2.945GiB / 3GiB 98.17% 956GB / 95.2GB 39.5GB / 41.7GB 934 dde7576a483f bvq-mongodb 16.69% 3.6GiB / 4GiB 90.00% 428GB / 2.22TB 987GB / 2.94TB 57 5d647c102f8d bvq-traefik 0.02% 49.53MiB / 27.27GiB 0.18% 1.16GB / 1.16GB 3.94GB / 0B 14 019eecba538d bvq-server-BVQ-16763-itsm_service_now 13.78% 3.96GiB / 4GiB 98.99% 825GB / 168GB 2.63GB / 4.1kB 215 5c13aa3c40cd portainer 0.02% 23.15MiB / 27.27GiB 0.08% 29.3kB / 0B 1.3GB / 756MB 11 aca7c41b0927 bvq-mongodb-BVQ-16763-itsm_service_now 4.89% 2.918GiB / 4GiB 72.95% 150GB / 825GB 115GB / 1.11TB 60 62b921699641 bvq-scanner-BVQ-16763-itsm_service_now 12.63% 1.878GiB / 2GiB 93.88% 374GB / 36.1GB 3.72GB / 33.8GB 303 57498d399f14 logging_filebeat_1 1.04% 111.3MiB / 27.27GiB 0.40% 174MB / 7.71GB 3.62GB / 2.49MB 15