diff --git a/dev-support/README.md b/dev-support/README.md index 98222b94d..4c45fbac9 100644 --- a/dev-support/README.md +++ b/dev-support/README.md @@ -32,25 +32,24 @@ yum install -y git docker ```shell git clone https://github.com/apache/bigtop-manager.git ``` -> You need to change the `node.version` in the `pom.xml` file under the `bigtop-manager-ui` module to `16.x` if you are using CentOS-7. ### **Step 3**: Build develop basic image -Run the setup command, you will get `bigtop-manager/develop:trunk-rocky-8` image. It has the environment needed to compile Bigtop-Manager and run servers such as Bigtop-Manager Server, Bigtop-Manager Agent, Mysql, etc. +Run the setup command, you will get `bigtop-manager/develop:rocky-8` image. It has the environment needed to compile Bigtop-Manager and run servers such as Bigtop-Manager Server, Bigtop-Manager Agent, Mysql, etc. **RHEL (Rocky 8) :** ```shell -/bin/bash dev-support/docker/image/build.sh trunk-rocky-8 +/bin/bash dev-support/docker/image/build.sh rocky-8 ``` ### **Step 4**: Build source & create cluster -* Bigtop Manager UI、Bigtop Manager Server Debug Port、MariaDB Server are also exposed to local ports: 8080、5005、3306. -* Docker hostnames are: bigtop-manager-server、bigtop-manager-agent-01、bigtop-manager-agent-02. +* Bigtop Manager UI、Bigtop Manager Server Debug Port、MariaDB Server are also exposed to local ports: 18080、15005、13306. +* Docker hostnames are: bm-1、bm-2、bm-3 and etc. **RHEL (Rocky 8) :** ```shell -/bin/bash dev-support/docker/containers/build.sh -e postgres -c 3 -o trunk-rocky-8 [--skip-compile] +/bin/bash dev-support/docker/containers/build.sh -e postgres -c 3 -o rocky-8 [--skip-compile] ``` ### **Step 5**: Access Web UI -Now you can access Web UI which exposes on `http://localhost:8080`. Log in with username `admin` and password `admin`. +Now you can access Web UI which exposes on `http://localhost:18080`. Log in with username `admin` and password `admin`. ### **Step 6**: Clear cluster Clean up the containers when you are done developing or testing. diff --git a/dev-support/docker/containers/build.sh b/dev-support/docker/containers/build.sh index 4fc0f4cc2..5cfce3ac4 100644 --- a/dev-support/docker/containers/build.sh +++ b/dev-support/docker/containers/build.sh @@ -20,7 +20,7 @@ usage() { echo " commands:" echo " -c NUM_INSTANCES, --create NUM_INSTANCES - Create Docker containers based bigtop-manager cluster, defaults to 3" echo " -e, --database - The specified database, defaults to postgres" - echo " -o, --os - Specify the operating system, default is trunk-rocky-8" + echo " -o, --os - Specify the operating system, default is rocky-8" echo " --skip-compile - Skip Compile" echo " -d, --destroy - Destroy all containers" echo " -h, --help" @@ -32,8 +32,8 @@ log() { } build() { - log "Build on docker $SKIP_BUILD" - if ! $SKIP_BUILD; then + log "Build on docker: $SKIP_COMPILE" + if ! $SKIP_COMPILE; then log "Compiling bigtop-manager" docker run -it --rm -u $(id -u):$(id -g) \ -v $PWD/../../../:/opt/develop/bigtop-manager/ \ @@ -67,21 +67,20 @@ create_container() { log "Create ${container_name}" if [ $i -eq 1 ]; then docker run -itd -p 15005:5005 -p 15006:5006 -p 18080:8080 --name ${container_name} --hostname ${container_name} --network bigtop-manager --cap-add=SYS_TIME bigtop-manager/develop:${OS} - docker cp ../../../bigtop-manager-server/target/bigtop-manager-server.tar.gz ${container_name}:/opt/ + docker cp ../../../bigtop-manager-dist/target/apache-bigtop-manager-*-server.tar.gz ${container_name}:/opt/bigtop-manager-server.tar.gz docker exec ${container_name} bash -c "cd /opt && tar -zxvf bigtop-manager-server.tar.gz" + docker exec ${container_name} bash -c "ssh-keygen -f '/root/.ssh/id_rsa' -N '' -t rsa" SERVER_PUB_KEY=`docker exec ${container_name} /bin/cat /root/.ssh/id_rsa.pub` else docker run -itd --name ${container_name} --hostname ${container_name} --network bigtop-manager --cap-add=SYS_TIME bigtop-manager/develop:${OS} fi - docker cp ../../../bigtop-manager-agent/target/bigtop-manager-agent.tar.gz ${container_name}:/opt/ - docker exec ${container_name} bash -c "cd /opt && tar -zxvf bigtop-manager-agent.tar.gz" - docker exec ${container_name} bash -c "echo '$SERVER_PUB_KEY' > /root/.ssh/authorized_keys" - docker exec ${container_name} ssh-keygen -N '' -t rsa -b 2048 -f /etc/ssh/ssh_host_rsa_key - docker exec ${container_name} ssh-keygen -N '' -t ecdsa -b 256 -f /etc/ssh/ssh_host_ecdsa_key - docker exec ${container_name} ssh-keygen -N '' -t ed25519 -b 256 -f /etc/ssh/ssh_host_ed25519_key - docker exec ${container_name} /bin/systemctl start sshd - - docker exec ${container_name} bash -c "systemctl start chronyd && chronyc tracking" + + docker exec ${container_name} bash -c "echo '$SERVER_PUB_KEY' > /root/.ssh/authorized_keys" + docker exec ${container_name} ssh-keygen -N '' -t rsa -b 2048 -f /etc/ssh/ssh_host_rsa_key + docker exec ${container_name} ssh-keygen -N '' -t ecdsa -b 256 -f /etc/ssh/ssh_host_ecdsa_key + docker exec ${container_name} ssh-keygen -N '' -t ed25519 -b 256 -f /etc/ssh/ssh_host_ed25519_key + docker exec ${container_name} /bin/systemctl start sshd + docker exec ${container_name} bash -c "systemctl start chronyd && chronyc tracking" done containers=($(docker network inspect bigtop-manager -f '{{range .Containers}}{{.Name}}{{" "}}{{end}}')) @@ -109,14 +108,12 @@ create_container() { docker exec ${container} bash -c "sed -i 's/localhost:5432/bm-mysql:3306/' /opt/bigtop-manager-server/conf/application.yml" docker exec ${container} bash -c "sed -i 's/postgres/root/' /opt/bigtop-manager-server/conf/application.yml" elif [ $DATABASE == "postgres" ]; then - docker exec ${container} bash -c "yum install postgresql -y" docker exec ${container} bash -c "PGPASSWORD=postgres psql -h bm-postgres -p5432 -U postgres -c 'create database bigtop_manager'" docker exec ${container} bash -c "PGPASSWORD=postgres psql -h bm-postgres -p5432 -U postgres -d bigtop_manager -f /opt/bigtop-manager-server/ddl/PostgreSQL-DDL-CREATE.sql" docker exec ${container} bash -c "sed -i 's/localhost:5432/bm-postgres:5432/' /opt/bigtop-manager-server/conf/application.yml" fi docker exec ${container} bash -c "nohup /bin/bash /opt/bigtop-manager-server/bin/start.sh --debug > /dev/null 2>&1 &" fi - docker exec ${container} bash -c "nohup /bin/bash /opt/bigtop-manager-agent/bin/start.sh --debug > /dev/null 2>&1 &" log "All Service Started!!!" done @@ -125,7 +122,7 @@ create_container() { create_db() { if [ $DATABASE == "mysql" ]; then docker run --restart=always -it -d \ - -p 3306:3306 \ + -p 13306:3306 \ --cap-add=SYS_TIME \ --network bigtop-manager \ --name bm-mysql \ @@ -150,7 +147,7 @@ create_db() { done elif [ $DATABASE == "postgres" ]; then docker run --restart=always -d \ - -p 5432:5432 \ + -p 15432:5432 \ --network bigtop-manager \ --name bm-postgres \ --hostname bm-postgres \ @@ -167,9 +164,9 @@ echo $PWD PROG=`basename $0` DATABASE=postgres -OS=trunk-rocky-8 +OS=rocky-8 NUM_INSTANCES=3 -SKIP_BUILD=false +SKIP_COMPILE=false while [ $# -gt 0 ]; do case "$1" in @@ -189,8 +186,8 @@ while [ $# -gt 0 ]; do echo "Requires a os" 1>&2 usage fi - if [ $2 != "trunk-rocky-8" ] && [ $2 != "trunk-centos-7" ] && [ $2 != "trunk-openeuler-22" ]; then - echo "The OS should be [trunk-rocky-8], [trunk-centos-7], or [trunk-openeuler-22]" 1>&2 + if [ $2 != "rocky-8" ] && [ $2 != "openeuler-24" ]; then + echo "The OS should be [rocky-8] or [openeuler-24]" 1>&2 usage fi OS=$2 @@ -207,7 +204,7 @@ while [ $# -gt 0 ]; do NUM_INSTANCES=$2 shift 2;; --skip-compile) - SKIP_BUILD=true + SKIP_COMPILE=true shift;; -d|--destroy) destroy @@ -221,7 +218,7 @@ while [ $# -gt 0 ]; do esac done -log "DATABASE: $DATABASE; OS: $OS; NUM_INSTANCES: $NUM_INSTANCES; SKIP_BUILD: $SKIP_BUILD; " +log "DATABASE: $DATABASE; OS: $OS; NUM_INSTANCES: $NUM_INSTANCES; SKIP_COMPILE: $SKIP_COMPILE; " build diff --git a/dev-support/docker/image/Dockerfile.openeuler24 b/dev-support/docker/image/Dockerfile.openeuler24 new file mode 100644 index 000000000..f8d8b68a9 --- /dev/null +++ b/dev-support/docker/image/Dockerfile.openeuler24 @@ -0,0 +1,29 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +FROM openeuler/openeuler:24.03 + +RUN dnf install -y sudo wget openssh-clients openssh-server mariadb mariadb-server net-tools chrony krb5-server krb5-libs krb5-workstation git python3 procps-ng +RUN dnf install -y postgresql java-17-openjdk-devel vim + +ENV JAVA_HOME=/usr/lib/jvm/java-17-openjdk +ENV PATH=$JAVA_HOME/bin:$PATH +RUN wget --no-check-certificate https://downloads.apache.org/maven/maven-3/3.8.8/binaries/apache-maven-3.8.8-bin.tar.gz && \ + tar -xvf apache-maven-3.8.8-bin.tar.gz && \ + mv apache-maven-3.8.8 /usr/lib/maven +ENV PATH=${PATH}:/usr/lib/maven/bin + +RUN wget https://raw.githubusercontent.com/gdraheim/docker-systemctl-replacement/master/files/docker/systemctl3.py -O /usr/bin/systemctl +RUN /bin/sed -i 's,# StrictHostKeyChecking ask,StrictHostKeyChecking no,g' /etc/ssh/ssh_config diff --git a/dev-support/docker/image/Dockerfile.template b/dev-support/docker/image/Dockerfile.rocky8 similarity index 89% rename from dev-support/docker/image/Dockerfile.template rename to dev-support/docker/image/Dockerfile.rocky8 index 1b3659489..978189a6c 100644 --- a/dev-support/docker/image/Dockerfile.template +++ b/dev-support/docker/image/Dockerfile.rocky8 @@ -13,11 +13,9 @@ # See the License for the specific language governing permissions and # limitations under the License. -FROM OS:VERSION +FROM rockylinux:8 -MAINTAINER dev@bigtop.apache.org - -RUN yum -y install sudo wget openssh-clients openssh-server vim mariadb mariadb-server net-tools chrony krb5-server krb5-libs krb5-workstation git rpm-build python3 procps-ng +RUN yum -y install sudo wget openssh-clients openssh-server vim postgresql mariadb mariadb-server net-tools chrony krb5-server krb5-libs krb5-workstation git rpm-build python3 procps-ng RUN wget https://download.java.net/java/GA/jdk17.0.2/dfd4a8d0985749f896bed50d7138ee7f/8/GPL/openjdk-17.0.2_linux-x64_bin.tar.gz -O /tmp/jdk.tar.gz --no-check-certificate \ && mkdir -p /usr/local/java && tar -xzf /tmp/jdk.tar.gz -C /usr/local/java --strip-components=1 \ && rm -f /tmp/jdk.tar.gz \ @@ -29,5 +27,3 @@ RUN wget https://archive.apache.org/dist/maven/maven-3/3.8.8/binaries/apache-mav RUN wget https://raw.githubusercontent.com/gdraheim/docker-systemctl-replacement/master/files/docker/systemctl3.py -O /usr/bin/systemctl RUN /bin/sed -i 's,# StrictHostKeyChecking ask,StrictHostKeyChecking no,g' /etc/ssh/ssh_config - -RUN ssh-keygen -f "/root/.ssh/id_rsa" -N "" diff --git a/dev-support/docker/image/build.sh b/dev-support/docker/image/build.sh index cc0d380d9..6f1238ad4 100644 --- a/dev-support/docker/image/build.sh +++ b/dev-support/docker/image/build.sh @@ -33,46 +33,31 @@ echo $PWD if [ $# != 1 ]; then echo "Creates bigtop-manager/develop image" echo - echo "Usage: build.sh " + echo "Usage: build.sh " echo - echo "Example: build.sh trunk-rocky-8" - echo " : build.sh 1.0.0-rocky-8" + echo "Example: build.sh rocky-8" exit 1 fi -PREFIX=$(echo "$1" | cut -d '-' -f 1) -OS=$(echo "$1" | cut -d '-' -f 2) -VERSION=$(echo "$1" | cut -d '-' -f 3-) +OS=$(echo "$1" | cut -d '-' -f 1) +VERSION=$(echo "$1" | cut -d '-' -f 2-) # Decimals are not supported. Either use integers only # e.g. 16.04 -> 16 VERSION_INT=$(echo "$VERSION" | cut -d '.' -f 1) -log "PREFIX: ${PREFIX}; OS: ${OS}; VERSION: ${VERSION}; VERSION_INT: ${VERSION_INT}" -DOCKER_OS=${OS} -DOCKER_VERSION=${VERSION} -CUSTOM_REPO="" +log "OS: ${OS}; VERSION: ${VERSION}; VERSION_INT: ${VERSION_INT}" case ${OS}-${VERSION_INT} in - centos-7) - CUSTOM_REPO="RUN mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.back \ - && curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo" - ;; rocky-8) - DOCKER_OS=rockylinux + DOCKERFILE="Dockerfile.rocky8" ;; - openeuler-22) - DOCKER_OS=openeuler/openeuler - DOCKER_VERSION=22.03 + openeuler-24) + DOCKERFILE="Dockerfile.openeuler24" ;; *) echo "Unsupported OS ${OS}-${VERSION}." exit 1 esac -# generate Dockerfile for build -sed -e "s|OS|${DOCKER_OS}|;s|VERSION|${DOCKER_VERSION}|" Dockerfile.template | - sed -e "/MAINTAINER dev@bigtop.apache.org/a\\$CUSTOM_REPO" > Dockerfile - -docker build --rm --no-cache -t bigtop-manager/develop:${PREFIX}-${OS}-${VERSION} -f Dockerfile ../.. -rm -f Dockerfile +docker build --rm --no-cache -t bigtop-manager/develop:${OS}-${VERSION} -f ${DOCKERFILE} ../.. diff --git a/docs/en/cluster.md b/docs/en/cluster.md index da0b64402..19a321953 100644 --- a/docs/en/cluster.md +++ b/docs/en/cluster.md @@ -17,6 +17,8 @@ When creating a cluster, users need to fill in the following information: ### Stack The stack page is a display page that mainly shows which optional services are available for subsequent installation. +Currently, the Bigtop stack provides tarball files generated by Apache Bigtop instead of rpm/deb packages. + ![Stack](https://github.com/user-attachments/assets/faf1112c-f0a6-4353-ba63-83abbb819c29) Expect for our official repository, users can also set their own repository which can configured here(due to license issue, MySQL/Grafana in Infra Stack are downloaded from the official website instead of the repository). diff --git a/docs/en/release.md b/docs/en/release.md index a83e60cb0..e91a88e1a 100644 --- a/docs/en/release.md +++ b/docs/en/release.md @@ -353,7 +353,7 @@ Hello Community, I am glad to announce that Apache Bigtop Manager 1.0.0 has been released. The source release can be downloaded here: -https://www.apache.org/dyn/closer.cgi/bigtop/bigtop-manager-1.0.0 +https://www.apache.org/dyn/closer.cgi/bigtop/bigtop-manager-1.0.0/ Detailed release notes can be checked here: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311420&version=12354831 diff --git a/docs/zh/cluster.md b/docs/zh/cluster.md index 3dab536c8..f13d05087 100644 --- a/docs/zh/cluster.md +++ b/docs/zh/cluster.md @@ -15,6 +15,8 @@ ### 组件栈 组件栈页面为展示页面,主要显示后续安装服务时有哪些可选服务 +目前 Bigtop 组件栈提供的是 Apache Bigtop 生成的 tarball 文件而非 rpm/deb 包 + ![Stack](https://github.com/user-attachments/assets/faf1112c-f0a6-4353-ba63-83abbb819c29) 除了官方 Repository 外,用户也可以搭建自己的 Repository 并在此处进行配置(注意 Infra 中的 MySQL/Grafana 由于 License 原因,是从官方网站下载而不是从 Repository 下载) diff --git a/docs/zh/release.md b/docs/zh/release.md index 67cfe7511..e2493be92 100644 --- a/docs/zh/release.md +++ b/docs/zh/release.md @@ -353,7 +353,7 @@ Hello Community, I am glad to announce that Apache Bigtop Manager 1.0.0 has been released. The source release can be downloaded here: -https://www.apache.org/dyn/closer.cgi/bigtop/bigtop-manager-1.0.0 +https://www.apache.org/dyn/closer.cgi/bigtop/bigtop-manager-1.0.0/ Detailed release notes can be checked here: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311420&version=12354831