[WIP] Platform/oci golden vm #13
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1 @@ | ||
| build*/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| # Notes for deploying on cloud-hypervisor | ||
|
|
||
| No notes yet, but we will want to be able to deploy the golden image (possibly | ||
| from an Oracle object path) via cloud-hypervisor. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| # Notes for building and pushing to OCI directly from VirtualBox | ||
|
|
||
| ## Blogs describe how to move an image from VBox > OCI | ||
|
|
||
| https://www.oracle.com/us/technologies/virtualization/oracle-vm-vb-oci-export-20190502-5480003.pdf | ||
|
|
||
| Actually, if you go far enough in this blog article (part 8): | ||
|
|
||
| https://blogs.oracle.com/virtualization/post/export-vm-from-virtualbox-to-oci-part-8 | ||
|
|
||
| He gives the command: | ||
| VBoxManage export <machine> | ||
| --output OCI:// | ||
| [--vsys <number of virtual system>] | ||
| [--vmname <name>] | ||
| [--cloud <number of virtual system>] | ||
| [--vmname <name>] | ||
| [--cloudprofile <cloud profile name>] | ||
| [--cloudbucket <bucket name>] | ||
| [--cloudkeepobject <true/false>] | ||
| [--cloudlaunchmode EMULATED|PARAVIRTUALIZED] | ||
| [--cloudlaunchinstance <true/false>] | ||
| [--clouddomain <domain>] | ||
| [--cloudshape <shape>] | ||
| [--clouddisksize <disk size in GB>] | ||
| [--cloudocivcn <OCI vcn id>] | ||
| [--cloudocisubnet <OCI subnet id>] | ||
| [--cloudpublicip <true/false>] | ||
| [--cloudprivateip <ip>] | ||
| [--cloudinitscriptpath <script path>] | ||
|
|
||
| ## Move the packer golden image | ||
|
|
||
| https://blogs.oracle.com/cloud-infrastructure/post/using-packer-and-virtualbox-to-bring-your-own-image-into-oracle-cloud-infrastructure | ||
|
|
||
| ## manual mode | ||
|
|
||
| Install the oci command line client | ||
|
|
||
| OSX | ||
|
|
||
| ```sh | ||
| brew install oci-cli | ||
| ``` | ||
|
|
||
| Linux | ||
|
|
||
| ```sh | ||
| python3 -m pip install oci-cli | ||
| ``` | ||
|
|
||
| Windows | ||
|
|
||
| ```sh | ||
| choclatey install oci-cli | ||
| ``` | ||
|
|
||
| configure with an OCI generated API key (gotta get that from the console: `https://cloud.oracle.com/identity/domains/my-profile/api-keys?region=us-phoenix-1`). Don't foget the public key fingerprint as well. | ||
|
|
||
| Download the .pem file that is generated (don't forget to do this, or you'll just need to create a new key). Store it in the ~/.oci directory (create if it doesn'[t exist). | ||
| ]) | ||
| Also, you need your tenancy, user, and compartment OCIDs. You | ||
|
|
||
| add them to: | ||
|
|
||
| ```sh | ||
| cat >> ~/.oci/config <<EOF | ||
| [DEFAULT] | ||
| user=<USER_OCID> | ||
| tenancy=<TENANT_OCID> | ||
| region=us-phoenix-1 | ||
| key_file=~/.oci/<API_AUTH_PRIVATE_KEY>.pem | ||
| fingerprint=<API_AUTH_PUB_FINGERPRINT> | ||
| EOF | ||
|
|
||
| cat >> ~/.oci/oci_cli_rc <<EOF | ||
| [DEFAULT] | ||
| compartment-id=<COMPARTMENT_ID> | ||
| EOF | ||
|
|
||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| - hosts: all | ||
|
Member
There was a problem hiding this comment. strange, I think this shouldn't be here. |
||
| any_errors_fatal: yes | ||
| become: true | ||
|
|
||
| tasks: | ||
| - import_role: | ||
| name: packages | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| - name: "Get installed packages" | ||
|
|
||
| apt: | ||
| list: "installed" | ||
| register: installed_packages | ||
|
|
||
| - name: "Install missing packages" | ||
| package: | ||
| state: "present" | ||
| name: "{{ item }}" | ||
| with_items: "{{ install_package_list | difference(installed_packages | json_query('results[*].name')) }}" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,157 @@ | ||
| --- | ||
|
Member
There was a problem hiding this comment. this looks like a rebase gone wrong. Probably as a result of my force push error to main... |
||
| install_package_list: | ||
| - apport | ||
| - bc | ||
| - bcache-tools | ||
| - binutils | ||
| - binutils-common | ||
| - binutils-x86-64-linux-gnu | ||
| - bolt | ||
| - btrfs-progs | ||
| - busybox-initramfs | ||
| - cryptsetup | ||
| - cryptsetup-bin | ||
| - dbus-user-session | ||
| - dirmngr | ||
| - distro-info | ||
| - dmeventd | ||
| - dosfstools | ||
| - dpkg-sig | ||
| - efibootmgr | ||
| - ethtool | ||
| - finalrd | ||
| - fonts-ubuntu-console | ||
| - fuse3 | ||
| - fwupd | ||
| - fwupd-signed | ||
| - gawk | ||
| - gcc-12-base | ||
| - gir1.2-glib-2.0 | ||
| - gir1.2-packagekitglib-1.0 | ||
| - gnupg | ||
| - gnupg-l10n | ||
| - gnupg-utils | ||
| - gpg | ||
| - gpg-agent | ||
| - gpgconf | ||
| - gpgsm | ||
| - gpg-wks-client | ||
| - gpg-wks-server | ||
| - grub-efi-amd64-bin | ||
| - grub-efi-amd64-signed | ||
| - grub-gfxpayload-lists | ||
| - htop | ||
| - initramfs-tools-bin | ||
| - iptables | ||
| - iptables-persistent | ||
| - keyutils | ||
| - kpartx | ||
| - libaio1 | ||
| - libarchive13 | ||
| - libassuan0 | ||
| - libatasmart4 | ||
| - libatm1 | ||
| - libbinutils | ||
| - libblockdev2 | ||
| - libblockdev-crypto2 | ||
| - libblockdev-fs2 | ||
| - libblockdev-loop2 | ||
| - libblockdev-part2 | ||
| - libblockdev-part-err2 | ||
| - libblockdev-swap2 | ||
| - libblockdev-utils2 | ||
| - libcbor0.8 | ||
| - libconfig-file-perl | ||
| - libctf0 | ||
| - libctf-nobfd0 | ||
| - libcurl3-gnutls | ||
| - libdevmapper-event1.02.1 | ||
| - libevent-core-2.1-7 | ||
| - libffi8 | ||
| - libfuse3-3 | ||
| - libfwupd2 | ||
| - libfwupdplugin5 | ||
| - libgcab-1.0-0 | ||
| - libgdbm6 | ||
| - libgdbm-compat4 | ||
| - libgirepository-1.0-1 | ||
| - libglib2.0-0 | ||
| - libglib2.0-data | ||
| - libgpgme11 | ||
| - libgpm2 | ||
| - libgudev-1.0-0 | ||
| - libgusb2 | ||
| - libicu70 | ||
| - libintl-perl | ||
| - libintl-xs-perl | ||
| - libip6tc2 | ||
| - libisns0 | ||
| - libjcat1 | ||
| - libjson-glib-1.0-0 | ||
| - libjson-glib-1.0-common | ||
| - libksba8 | ||
| - libldap-2.5-0 | ||
| - liblvm2cmd2.03 | ||
| - liblzo2-2 | ||
| - libmagic1 | ||
| - libmagic-mgc | ||
| - libmbim-glib4 | ||
| - libmbim-proxy | ||
| - libmm-glib0 | ||
| - libmodule-find-perl | ||
| - libmodule-scandeps-perl | ||
| - libmpfr6 | ||
| - libnetfilter-conntrack3 | ||
| - libnetplan0 | ||
| - libnfnetlink0 | ||
| - libnfsidmap1 | ||
| - libnl-3-200 | ||
| - libnl-genl-3-200 | ||
| - libnpth0 | ||
| - libnspr4 | ||
| - libnss3 | ||
| - libnss-systemd | ||
| - libntfs-3g89 | ||
| - libopeniscsiusr | ||
| - libpackagekit-glib2-18 | ||
| - libpam-cap | ||
| - libparted2 | ||
| - libparted-fs-resize0 | ||
| - libperl5.34 | ||
| - libpolkit-agent-1-0 | ||
| - libpolkit-gobject-1-0 | ||
| - libproc-processtable-perl | ||
| - libpython3.10-minimal | ||
| - libpython3.10-stdlib | ||
| - libqmi-glib5 | ||
| - libqmi-proxy | ||
| - libsemanage2 | ||
| - libsepol2 | ||
| - libsgutils2-2 | ||
| - libsigsegv2 | ||
| - libsmbios-c2 | ||
| - libsort-naturally-perl | ||
| - libssh-4 | ||
| - libssl3 | ||
| - libtcl8.6 | ||
| - libterm-readkey-perl | ||
| - libtss2-esys-3.0.2-0 | ||
| - libtss2-mu0 | ||
| - libtss2-sys1 | ||
| - libtss2-tcti-cmd0 | ||
| - libtss2-tcti-device0 | ||
| - libtss2-tcti-mssim0 | ||
| - libtss2-tcti-swtpm0 | ||
| - libudisks2-0 | ||
| - libusb-1.0-0 | ||
| - libvolume-key1 | ||
| - libxml2 | ||
| - libxmlb2 | ||
| - linux-headers-5.15.0-1033-oracle | ||
| - linux-headers-oracle | ||
| - linux-image-5.15.0-1033-oracle | ||
| - linux-image-oracle | ||
| - linux-modules-5.15.0-1033-oracle | ||
| - linux-modules-extra-5.15.0-1033-oracle | ||
| - linux-oracle | ||
| - linux-oracle-headers-5.15.0-1033 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| #!/bin/sh | ||
sdake marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| cat > /tmp/fixes.sh <<EOF | ||
| #!/bin/bash | ||
| set -x | ||
| echo 'GRUB_CMDLINE_LINUX="console=ttyS0"' >> /etc/default/grub | ||
|
Member
There was a problem hiding this comment. better to put in /etc/default/grub.d. I will submit an update.
Member
Author
There was a problem hiding this comment. This may not even work, still trying to get a testable environment for dev (I think I can do this in OCI now, but I haven't yet managed to get a connection through the UI or via |
||
| update-grub | ||
|
|
||
| if [ -f /etc/udev/rules.d/70-persistent-net.rules ] ; then | ||
| rm /etc/udev/rules.d/70-persistent-net.rules | ||
| ln -s /dev/null /etc/udev/rules.d/70-persistent-net.rules | ||
| fi | ||
|
|
||
| echo 'packer ALL=(ALL) NOPASSWD:ALL'>> /etc/sudoers.d/91-packer | ||
|
|
||
| passwd -d packer | ||
|
Member
There was a problem hiding this comment. I had to read the man page, and I was still not sure what in this case, it appears to have expired my password, which I guess is reasonable. Is the goal your really after to remove the ability to login with the password (but with ssh?) In that case, I think you want from man page:
Member
There was a problem hiding this comment. @rstarmer this needs a change to resolve. I recommend
Member
Author
There was a problem hiding this comment. Yes, that makes sense. I looked at the passwd/shadow files, and it "erases" the password, which would presumably make it impossible to login via a password, but would not expressly prohibit setting a password. And with passwordless Sudo, you could potentially set a password via
Member
There was a problem hiding this comment. this is important, but not worth slowing down the PR. Can you file a tech debt bug? |
||
|
|
||
| mkdir /home/packer/.ssh | ||
| chmod 700 /home/packer/.ssh | ||
| curl -sLo - https://github.com/sdake.keys >> /home/packer/.ssh/authorized_keys | ||
| curl -sLo - https://github.com/rstarmer.keys >> /home/packer/.ssh/authorized_keys | ||
|
Member
There was a problem hiding this comment. I am not really sure how we should handle dynamic secret management. For now, I have started a wiki entry so we know where our secrets are stored or consumed in our repo.
Member
Author
There was a problem hiding this comment. I think we can enable this as a provisioning step. If we're goign to use Ansible, then it would likely be easier to add an ansible provisioner that loads the right keys at build time, or even better at image deploy time so that there is nothing static in the base/golden image.
Member
There was a problem hiding this comment. Right. I think some things should be dynamic, and the rest should be baked (with packer). Dynamically loaded credentials make sense, far more sense then static. But, I spent like 5+ years working with people at Red Hat that battled and battled over this credential issue. My recommendation - tech debt it for now, we will sort out the workflow later. Another way to introduce SSH credentials is via the kernel command line. |
||
| EOF | ||
|
|
||
| echo ${SSH_PASSWORD} | sudo -S bash /tmp/fixes.sh | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| if [ $(oci os bucket list | jq '.data[0].name' | grep "debian-golden" | wc -l) -ge 1 ] ; then | ||
| echo "bucket debian-golden exists" | ||
| else | ||
| oci os bucket create --name debian-golden | ||
| if [ $? -gt 0 ]; then exit 1; fi | ||
| fi | ||
| if [ -f build/golden.raw-disk001.vmdk ]; then | ||
| oci os object put --bucket-name debian-golden --file build/golden.raw-disk001.vmdk --name golden.raw-disk001.vmdk --force | ||
| if [ $? -gt 0 ]; then exit 1; fi | ||
| else | ||
| echo no vmdk to upload! | ||
| exit 1 | ||
| fi | ||
| NAMESPACE=$(oci os ns get | jq .data | tr -d '"') | ||
| WORK_REQUEST=$(oci compute image import from-object --namespace $NAMESPACE \ | ||
| --launch-mode PARAVIRTUALIZED --display-name debian-golden \ | ||
| --bucket-name debian-golden --name golden.raw-disk001.vmdk ) | ||
| echo $WORK_REQUEST | ||
| WORK_REQUEST_ID=$(echo $WORK_REQUEST | jq '."opc-work-request-id"' | tr -d '"') | ||
| echo $WORK_REQUEST_ID | ||
| WORK_DISPLAY_NAME=$(echo $WORK_REQUEST | jq '.data."display-name"' | tr -d '"') | ||
| echo $WORK_DISPLAY_NAME | ||
| if [ $? -gt 0 ]; then exit 1; fi | ||
| time | ||
| while [ "$(oci work-requests work-request get --work-request-id $WORK_REQUEST_ID | jq '.data.status' >&/dev/null; echo $?)" -eq 0 ]; do echo waiting 1m; sleep 60; done | ||
| echo "SUCCESS importing $WORK_DISPLAY_NAME" |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/usr/sbinnot in the root path? Doesn't matter for the purposes of merging the PR, just seems odd. :)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@rstarmer can you respond on these questions? I am trying to learn along the way (this feels wrong, which means
/etc/environmentmay not be right).There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was to resolve an issue with running shutdown, and I'm not sure why the environment worked initially, and then didn't. I'll do a bit more debugging, and I agree, I've not seen a single example where the full path to shutdown was required.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
all good then. Perhaps a debt tech bug would help us keep track of it for new developer low hanging fruit.