Longhorn: Upgrade from 1.6.4. to 1.7.3
Change-Id: I4e713a453a6b9e983685e2db550066fd2694609f
diff --git a/charts/longhorn-1.6.4/ocp-readme.md b/charts/longhorn-1.6.4/ocp-readme.md
new file mode 100644
index 0000000..d2fb98e
--- /dev/null
+++ b/charts/longhorn-1.6.4/ocp-readme.md
@@ -0,0 +1,177 @@
+# OpenShift / OKD Extra Configuration Steps
+
+- [OpenShift / OKD Extra Configuration Steps](#openshift--okd-extra-configuration-steps)
+ - [Notes](#notes)
+ - [Known Issues](#known-issues)
+ - [Preparing Nodes (Optional)](#preparing-nodes-optional)
+ - [Default /var/lib/longhorn setup](#default-varliblonghorn-setup)
+ - [Separate /var/mnt/longhorn setup](#separate-varmntlonghorn-setup)
+ - [Create Filesystem](#create-filesystem)
+ - [Mounting Disk On Boot](#mounting-disk-on-boot)
+ - [Label and Annotate Nodes](#label-and-annotate-nodes)
+ - [Example values.yaml](#example-valuesyaml)
+ - [Installation](#installation)
+ - [Refs](#refs)
+
+## Notes
+
+Main changes and tasks for OCP are:
+
+- On OCP / OKD, the Operating System is Managed by the Cluster
+- OCP Imposes [Security Context Constraints](https://docs.openshift.com/container-platform/4.11/authentication/managing-security-context-constraints.html)
+ - This requires everything to run with the least privilege possible. For the moment every component has been given access to run as higher privilege.
+ - Something to circle back on is network polices and which components can have their privileges reduced without impacting functionality.
+ - The UI probably can be for example.
+- openshift/oauth-proxy for authentication to the Longhorn Ui
+ - **⚠️** Currently Scoped to Authenticated Users that can delete a longhorn settings object.
+ - **⚠️** Since the UI it self is not protected, network policies will need to be created to prevent namespace <--> namespace communication against the pod or service object directly.
+ - Anyone with access to the UI Deployment can remove the route restriction. (Namespace Scoped Admin)
+- Option to use separate disk in /var/mnt/longhorn & MachineConfig file to mount /var/mnt/longhorn
+- Adding finalizers for mount propagation
+
+## Known Issues
+
+- General Feature/Issue Thread
+ - [[FEATURE] Deploying Longhorn on OKD/Openshift](https://github.com/longhorn/longhorn/issues/1831)
+- 4.10 / 1.23:
+ - 4.10.0-0.okd-2022-03-07-131213 to 4.10.0-0.okd-2022-07-09-073606
+ - Tested, No Known Issues
+- 4.11 / 1.24:
+ - 4.11.0-0.okd-2022-07-27-052000 to 4.11.0-0.okd-2022-11-19-050030
+ - Tested, No Known Issues
+ - 4.11.0-0.okd-2022-12-02-145640, 4.11.0-0.okd-2023-01-14-152430:
+ - Workaround: [[BUG] Volumes Stuck in Attach/Detach Loop](https://github.com/longhorn/longhorn/issues/4988)
+ - [MachineConfig Patch](https://github.com/longhorn/longhorn/issues/4988#issuecomment-1345676772)
+- 4.12 / 1.25:
+ - 4.12.0-0.okd-2022-12-05-210624 to 4.12.0-0.okd-2023-01-20-101927
+ - Tested, No Known Issues
+ - 4.12.0-0.okd-2023-01-21-055900 to 4.12.0-0.okd-2023-02-18-033438:
+ - Workaround: [[BUG] Volumes Stuck in Attach/Detach Loop](https://github.com/longhorn/longhorn/issues/4988)
+ - [MachineConfig Patch](https://github.com/longhorn/longhorn/issues/4988#issuecomment-1345676772)
+ - 4.12.0-0.okd-2023-03-05-022504 - 4.12.0-0.okd-2023-04-16-041331:
+ - Tested, No Known Issues
+- 4.13 / 1.26:
+ - 4.13.0-0.okd-2023-05-03-001308 - 4.13.0-0.okd-2023-08-18-135805:
+ - Tested, No Known Issues
+- 4.14 / 1.27:
+ - 4.14.0-0.okd-2023-08-12-022330 - 4.14.0-0.okd-2023-10-28-073550:
+ - Tested, No Known Issues
+
+## Preparing Nodes (Optional)
+
+Only required if you require additional customizations, such as storage-less nodes, or secondary disks.
+
+### Default /var/lib/longhorn setup
+
+Label each node for storage with:
+
+```bash
+oc get nodes --no-headers | awk '{print $1}'
+
+export NODE="worker-0"
+oc label node "${NODE}" node.longhorn.io/create-default-disk=true
+```
+
+### Separate /var/mnt/longhorn setup
+
+#### Create Filesystem
+
+On the storage nodes create a filesystem with the label longhorn:
+
+```bash
+oc get nodes --no-headers | awk '{print $1}'
+
+export NODE="worker-0"
+oc debug node/${NODE} -t -- chroot /host bash
+
+# Validate Target Drive is Present
+lsblk
+
+export DRIVE="sdb" #vdb
+sudo mkfs.ext4 -L longhorn /dev/${DRIVE}
+```
+
+> ⚠️ Note: If you add New Nodes After the below Machine Config is applied, you will need to also reboot the node.
+
+#### Mounting Disk On Boot
+
+The Secondary Drive needs to be mounted on every boot. Save the Concents and Apply the MachineConfig with `oc apply -f`:
+
+> ⚠️ This will trigger an machine config profile update and reboot all worker nodes on the cluster
+
+```yaml
+apiVersion: machineconfiguration.openshift.io/v1
+kind: MachineConfig
+metadata:
+ labels:
+ machineconfiguration.openshift.io/role: worker
+ name: 71-mount-storage-worker
+spec:
+ config:
+ ignition:
+ version: 3.2.0
+ systemd:
+ units:
+ - name: var-mnt-longhorn.mount
+ enabled: true
+ contents: |
+ [Unit]
+ Before=local-fs.target
+ [Mount]
+ Where=/var/mnt/longhorn
+ What=/dev/disk/by-label/longhorn
+ Options=rw,relatime,discard
+ [Install]
+ WantedBy=local-fs.target
+```
+
+#### Label and Annotate Nodes
+
+Label and annotate storage nodes like this:
+
+```bash
+oc get nodes --no-headers | awk '{print $1}'
+
+export NODE="worker-0"
+oc annotate node ${NODE} --overwrite node.longhorn.io/default-disks-config='[{"path":"/var/mnt/longhorn","allowScheduling":true}]'
+oc label node ${NODE} node.longhorn.io/create-default-disk=config
+```
+
+## Example values.yaml
+
+Minimum Adjustments Required
+
+```yaml
+openshift:
+ oauthProxy:
+ repository: quay.io/openshift/origin-oauth-proxy
+ tag: 4.15 # Use Your OCP/OKD 4.X Version, Current Stable is 4.15
+
+# defaultSettings: # Preparing nodes (Optional)
+ # createDefaultDiskLabeledNodes: true
+
+openshift:
+ enabled: true
+ ui:
+ route: "longhorn-ui"
+ port: 443
+ proxy: 8443
+```
+
+## Installation
+
+```bash
+# helm template ./chart/ --namespace longhorn-system --values ./chart/values.yaml --no-hooks > longhorn.yaml # Local Testing
+helm template longhorn --namespace longhorn-system --values values.yaml --no-hooks > longhorn.yaml
+oc create namespace longhorn-system -o yaml --dry-run=client | oc apply -f -
+oc apply -f longhorn.yaml -n longhorn-system
+```
+
+## Refs
+
+- <https://docs.openshift.com/container-platform/4.11/storage/persistent_storage/persistent-storage-iscsi.html>
+- <https://docs.okd.io/4.11/storage/persistent_storage/persistent-storage-iscsi.html>
+- okd 4.5: <https://github.com/longhorn/longhorn/issues/1831#issuecomment-702690613>
+- okd 4.6: <https://github.com/longhorn/longhorn/issues/1831#issuecomment-765884631>
+- oauth-proxy: <https://github.com/openshift/oauth-proxy/blob/master/contrib/sidecar.yaml>
+- <https://github.com/longhorn/longhorn/issues/1831>