Advanced cluster management¶

Advanced cluster management with OSM client¶

This guide contains OSM client commands to operate infrastructure and applications following the new declarative framework introduced in Release SIXTEEN.

OSM client initialization¶

export OSM_HOSTNAME=$(kubectl get -n osm -o jsonpath="{.spec.rules[0].host}" ingress nbi-ingress)
echo "OSM_HOSTNAME: $OSM_HOSTNAME"

VIM/Cloud account operations¶

VIM/Cloud account registration¶

Azure Cloud account registration¶

export OSM_CREDS_FOLDER="${HOME}/vims"
source ${OSM_CREDS_FOLDER}/azure-env.rc
osm cloud-account-add --name azure-site --account_type azure \
    --auth_url http://www.azure.com \
    --user "$AZURE_CLIENT_ID" --password "$AZURE_SECRET" --tenant "$AZURE_TENANT"  \
    --description "AZURE site" \
    --creds ${OSM_CREDS_FOLDER}/azure-credentials.json \
    --config "{region_name: westeurope, resource_group: '<AZURE_RG>', subscription_id: '$AZURE_SUBSCRIPTION_ID', vnet_name: 'osm', flavors_pattern: '^Standard'}"

File ${OSM_CREDS_FOLDER}/azure-env.rc:

export AZURE_CLIENT_ID="**********************************"
export AZURE_TENANT="**********************************"
export AZURE_SECRET="**********************************"
export AZURE_SUBSCRIPTION_ID="**********************************"

File ${OSM_CREDS_FOLDER}/azure-credentials.json:

{
  "clientId": "{************************************}",
  "clientSecret": "************************************",
  "subscriptionId": "************************************",
  "tenantId": "************************************",
  "activeDirectoryEndpointUrl": "https://login.microsoftonline.com",
  "resourceManagerEndpointUrl": "https://management.azure.com/",
  "activeDirectoryGraphResourceId": "https://graph.windows.net/",
  "sqlManagementEndpointUrl": "https://management.core.windows.net:8443/",
  "galleryEndpointUrl": "https://gallery.azure.com/",
  "managementEndpointUrl": "https://management.core.windows.net/"
}

The JSON credentials file corresponds to the service principal credentials obtained during the service principal creation: https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/set-up-clusters-from-hosted-kubernetes-providers/aks

az ad sp create-for-rbac --role Contributor --scopes /subscriptions/<subscription-id>/resourceGroups/<resource-group-name>

Dummy Cloud account registration¶

If you have existing clusters, you can register them to a dummy cloud account. In order to register a dummy cloud account, you can proceed this way:

osm cloud-account-add --name mylocation1 \
    --user u --password p --tenant p \
    --account_type dummy \
    --auth_url http://localhost/dummy

VIM/Cloud account deletion¶

osm cloud-account-remove azure-site

Cluster operations¶

Cluster creation¶

Azure (AKS)¶

CLUSTER_NAME=cluster1
CLUSTER_VM_SIZE=Standard_D2_v2
CLUSTER_NODES=1
REGION_NAME=northeurope
VIM_ACCOUNT=azure-site
RESOURCE_GROUP=<AZURE_RG>
KUBERNETES_VERSION="1.30"
osm cluster-create --node-count ${CLUSTER_NODES} --node-size ${CLUSTER_VM_SIZE} --version ${KUBERNETES_VERSION} --vim-account ${VIM_ACCOUNT} --description "Cluster1" ${CLUSTER_NAME} --region-name ${REGION_NAME} --resource-group ${RESOURCE_GROUP}

OpenStack (CAPI)¶

CLUSTER_NAME=cluster1
CLUSTER_NODES=1
CLUSTER_VM_SIZE=m1.large
VIM_ACCOUNT=openstack-site
KUBERNETES_VERSION="1.30"
osm cluster-create --node-count ${CLUSTER_NODES} --node-size ${CLUSTER_VM_SIZE} --version ${KUBERNETES_VERSION} --vim-account ${VIM_ACCOUNT} --description "Cluster1" ${CLUSTER_NAME}

To add an additional OpenStack Network Security Group (NSG) to the virtual machines of the cluster, pass the security group ID or name via --config (or --config_file). Use only one of them; if both are given, the ID takes precedence:

# By security group ID
osm cluster-create ... --config "capi-openstack: {security-group-id: <SG_UUID>}" ${CLUSTER_NAME}

# By security group name
osm cluster-create ... --config "capi-openstack: {security-group-name: my-sg}" ${CLUSTER_NAME}

If neither security-group-id nor security-group-name is specified, the OpenStack project's default security group is applied, in addition to the security groups managed by the Cluster API OpenStack provider (CAPO).

osm cluster-list
osm cluster-show cluster1

When the cluster is created, the field resourceState should be READY.

Cluster update¶

Use cluster-update to change the configuration of an existing CAPI OpenStack cluster. The --config option accepts a YAML string with the capi-openstack section, and --config_file reads it from a file (the two options are mutually exclusive):

# Update the security group (by ID)
osm cluster-update cluster1 --config "capi-openstack: {security-group-id: <NEW_SG_UUID>}"

# Update the security group (by name)
osm cluster-update cluster1 --config "capi-openstack: {security-group-name: new-sg-name}"

# Load config from a YAML file
osm cluster-update cluster1 --config_file cluster-config.yaml

Example cluster-config.yaml:

capi-openstack:
  security-group-name: my-restricted-sg

The new configuration is stored with the cluster, so it is kept in later updates. The same fields (Security Group ID and Security Group Name) are available in the NG-UI when creating or updating an OpenStack cluster.

Getting kubeconfig¶

Once the cluster is ready, you can get the credentials in this way:

osm cluster-show cluster1 -o jsonpath='{.credentials}' | yq -P
# Save them in a file
osm cluster-show cluster1 -o jsonpath='{.credentials}' | yq -P > ~/kubeconfig-cluster1.yaml
# Test it
export KUBECONFIG=~/kubeconfig-cluster1.yaml
kubectl get nodes

In case credentials are renewed by the cloud policy, credentials can be obtained using this command:

osm cluster-get-credentials cluster1

Cluster scale¶

osm cluster-scale cluster1 --node-count 2

Cluster creation on OpenNebula¶

Kubernetes clusters can be created on an OpenNebula VIM/Cloud account (see How to add OpenNebula as VIM target in OSM) using Cluster API for OpenNebula (CAPONE). The cluster is created with osm cluster-create and the OpenNebula specific parameters are passed in the capi-opennebula section of --config:

CLUSTER_NAME=cluster-one
VIM_ACCOUNT=opennebula-site
osm cluster-create --node-count 2 --node-size default --version "1.30" --vim-account ${VIM_ACCOUNT} \
    --description "Cluster in OpenNebula" ${CLUSTER_NAME} \
    --config "capi-opennebula: {public-network-name: public, control-plane-host: 192.168.0.10, worker-image-name: osm-capone-node-1.30}"

The following parameters of the capi-opennebula section are available:

Parameter	Default	Description
`public-network-name`	(empty)	Name of the OpenNebula virtual network where the cluster VMs are connected
`control-plane-host`	(empty)	Address (host) of the control plane endpoint of the cluster
`worker-image-name`	`osm-capone-node-<k8s version>`	Name of the OpenNebula image used for the worker nodes
`control-plane-image-name`	same as `worker-image-name`	Name of the OpenNebula image used for the control plane nodes
`control_plane_node_count`	`1`	Number of control plane nodes
`control_plane_node_size`	same as `--node-size`	Size of the control plane nodes
`cluster-cni`	`calico`	CNI of the cluster

The size of the nodes (--node-size and control_plane_node_size) is expressed as CPU:VCPU:MEMORY_MB, for instance 2:4:8192 for 2 physical CPUs, 4 virtual CPUs and 8192 MB of memory. The value default (or an empty value) means 1:2:4096.

When the OpenNebula account is registered, OSM stores its credentials in a Secret named <account name>-one-credentials in the management cluster, encrypted with SOPS/age, so that the CAPONE provider can use them. The password is not written in clear text in any repository.

The cluster is ready when the field resourceState of osm cluster-show is READY. Cluster deletion is supported for OpenNebula clusters, whereas cluster update operations (scale, upgrade and configuration changes) are not supported yet.

Cluster deletion¶

osm cluster-delete cluster1

Cluster registration¶

This should be run over a cluster that was not created by OSM:

CLUSTER_NAME=cluster2
VIM_ACCOUNT=azure-site
osm cluster-register --creds ~/kubeconfig-${CLUSTER_NAME}.yaml --vim ${VIM_ACCOUNT} --description "My existing K8s cluster" ${CLUSTER_NAME}

osm cluster-list
osm cluster-show cluster2

When the cluster is created, the field resourceState should be READY.

Cluster deregistration¶

osm cluster-deregister cluster2

OKA operations¶

OKA addition¶

# git clone --recursive https://osm.etsi.org/gitlab/vnf-onboarding/osm-packages.git
export OSM_PACKAGES_FOLDER="${HOME}/osm-packages"
export OKA_FOLDER="${OSM_PACKAGES_FOLDER}/oka/apps"
osm oka-add jenkins ${OKA_FOLDER}/jenkins --description jenkins --profile-type app-profile
osm oka-add testapp ${OKA_FOLDER}/testapp --description testapp --profile-type app-profile
osm oka-add testacme ${OKA_FOLDER}/testacme --description testacme --profile-type app-profile

osm oka-list

When the OKA is created, the field resourceState should be READY.

OKA deletion¶

osm oka-delete testapp
osm oka-delete testacme
osm oka-delete jenkins

OKA generation for helm charts¶

osm oka-generate jenkins --base-directory okas --profile-type app-profile --helm-repo-name bitnamicharts --helm-repo-url oci://registry-1.docker.io/bitnamicharts --helm-chart jenkins --version 13.4.20 --namespace jenkins
tree okas/jenkins
# Once generated, you can add it with:
osm oka-add jenkins okas/jenkins --description jenkins --profile-type app-profile

Profile operations¶

Listing profiles¶

osm profile-list

KSU operations¶

KSU creation from OKA¶

You must specify the destination profile:

export OSM_PACKAGES_FOLDER="${HOME}/osm-packages"
export OKA_FOLDER="${OSM_PACKAGES_FOLDER}/oka/apps"
osm ksu-create --ksu testapp --profile mydemo --profile-type app-profile --oka testapp --params ${OKA_FOLDER}/testapp-params.yaml
osm ksu-create --ksu testacme --profile mydemo --profile-type app-profile --oka testacme --params ${OKA_FOLDER}/testacme-params.yaml
osm ksu-create --ksu jenkins --description "Jenkins" --profile mydemo --profile-type app-profile --oka jenkins --params ${OKA_FOLDER}/jenkins-params.yaml

osm ksu-list

When the KSU is created, the field resourceState should be READY.

KSU deletion¶

osm ksu-delete testapp
osm ksu-delete testacme
osm ksu-delete jenkins