我将 docker 私有注册表 v2.7.0 作为具有服务和持久卷的 kubernetes pod 运行，这要感谢 Varun Kumar G 教程，这是我设置中唯一成功的方法，让 kubernetes 从我的私有 docker 注册表中提取我的3 个节点 - 本地 - 带有 ubuntu 20.04 lts kvms 的集群。
问题在于从 kubernetes pod docker registry v2.7.0 中删除图像（必须使用以前的版本，因为最新的 v2.7.1 不适用于 htpasswd）。此外，我已经阅读了很多类似的主题，例如this、this和this。
使用 docker registry v2.7.1作为 docker 容器运行，我可以毫无问题地删除图像，
但是使用 docker registry v2.7.0作为 kubernetes pod运行，通常的删除步骤导致无法再次推送已删除的映像，即使在成功删除 blob 并手动删除/var/lib/registry/docker/registry/v2/repositories/.
下面是注册表 pod yaml
apiVersion: v1
kind: Pod
metadata:
name: dockreg-pod
labels:
app: mregistry
spec:
containers:
- name: registry
image: registry:2.7.0
imagePullPolicy: IfNotPresent
volumeMounts:
- name: repo-vol
mountPath: "/var/lib/registry"
- name: certs-vol
mountPath: "/certs"
readOnly: true
- name: auth-vol
mountPath: "/auth"
readOnly: true
env:
- name: REGISTRY_AUTH
value: "htpasswd"
- name: REGISTRY_AUTH_HTPASSWD_REALM
value: "Registry Realm"
- name: REGISTRY_AUTH_HTPASSWD_PATH
value: "/auth/htpasswd"
- name: REGISTRY_HTTP_TLS_CERTIFICATE
value: "/certs/tls.crt"
- name: REGISTRY_HTTP_TLS_KEY
value: "/certs/tls.key"
- name: REGISTRY_STORAGE_DELETE_ENABLED
value: "true"
volumes:
- name: repo-vol
persistentVolumeClaim:
claimName: repo-pvc
- name: certs-vol
secret:
secretName: certs-secret
- name: auth-vol
secret:
secretName: auth-secret
restartPolicy: Always
nodeName: spring
以下是持久卷 yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: repo-pv
labels:
type: prstore
spec:
capacity:
storage: 7Gi
volumeMode: Filesystem
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
storageClassName: local-storage
local:
fsType: ext4
path: /root/repo
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- spring
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: repo-pvc
labels:
type: prstore
spec:
selector:
matchLabels:
type: prstore
volumeMode: Filesystem
storageClassName: local-storage
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 7Gi
假设我在一个全新的注册表 pod 上推送一个图像，并且预先清除了持久性存储。
root@sea:scripts# docker push dockreg:5000/mubu4:v4
The push refers to repository [dockreg:5000/mubu4]
9f54eef41275: Pushed
v4: digest: sha256:7bd0d9a9821815dccb5c53c18cea04591ec633e2e529c5cdd39681169589c17f size: 529
删除图像似乎按预期工作，直到我再次尝试推送已删除的图像，此时我遇到了可怕的Layer already exists错误。
正如您可能在上面看到的，我在注册表 pod 环境中包含以下内容，
- name: REGISTRY_STORAGE_DELETE_ENABLED
value: "true"
否则我会unsupported从调用中得到一个错误curl -X DELETE，即使在添加之后
delete:
enabled: true
在/etc/docker/registry/config.yml吊舱内，
version: 0.1
log:
fields:
service: registry
storage:
cache:
blobdescriptor: inmemory
filesystem:
rootdirectory: /var/lib/registry
delete:
enabled: true
http:
addr: :5000
headers:
X-Content-Type-Options: [nosniff]
health:
storagedriver:
enabled: true
interval: 10s
threshold: 3
这似乎对我的用例没有影响。
以下是删除步骤。
curl -u alexander:sofianos \
> -vsk -H "Accept: \
> application/vnd.docker.distribution.manifest.v2+json" \
> -X DELETE \
> https://dockreg:5000/v2/mubu4/manifests/sha256:\
> 7bd0d9a9821815dccb5c53c18cea04591ec633e2e529c5cdd39681169589c17f
上面打印了以下内容
> DELETE /v2/mubu4/manifests/sha256:7bd0d9a9821815dccb5c53c18cea04591ec633e2e529c5cdd39681169589c17f HTTP/2
> Host: dockreg:5000
> authorization: Basic YWxleGFuZGVyOnNvZmlhbm9z
> user-agent: curl/7.68.0
> accept: application/vnd.docker.distribution.manifest.v2+json
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 202
< docker-distribution-api-version: registry/2.0
< x-content-type-options: nosniff
< content-length: 0
< date: Sat, 30 Oct 2021 13:25:53 GMT
<
* Connection #0 to host dockreg left intact
这似乎是有序的。
下面，从注册表 pod 中删除 blob
root@sea:scripts# kubectl exec -it dockreg-pod -- sh
/ # bin/registry garbage-collect /etc/docker/registry/config.yml
mubu4
0 blobs marked, 3 blobs and 0 manifests eligible for deletion
blob eligible for deletion: sha256:7b1a6ab2e44dbac178598dabe7cff59bd67233dba0b27e4fbd1f9d4b3c877a54
INFO[0000] Deleting blob: /docker/registry/v2/blobs/sha256/7b/7b1a6ab2e44dbac178598dabe7cff59bd67233dba0b27e4fbd1f9d4b3c877a54 go.version=go1.11.2 instance.id=82a101ee-47f4-4f4f-bc79-76d774b0924b service=registry
blob eligible for deletion: sha256:7bd0d9a9821815dccb5c53c18cea04591ec633e2e529c5cdd39681169589c17f
INFO[0000] Deleting blob: /docker/registry/v2/blobs/sha256/7b/7bd0d9a9821815dccb5c53c18cea04591ec633e2e529c5cdd39681169589c17f go.version=go1.11.2 instance.id=82a101ee-47f4-4f4f-bc79-76d774b0924b service=registry
blob eligible for deletion: sha256:ecb35fc8715f5ab1d9053ecb2f2d9ebbec4a59c0a0615d98de53bc29f7285085
INFO[0000] Deleting blob: /docker/registry/v2/blobs/sha256/ec/ecb35fc8715f5ab1d9053ecb2f2d9ebbec4a59c0a0615d98de53bc29f7285085 go.version=go1.11.2 instance.id=82a101ee-47f4-4f4f-bc79-76d774b0924b service=registry
最后，手动删除存储库映像
/ # rm -rf /var/lib/registry/docker/registry/v2/repositories/mubu4
在我的持久存储上，注册表现在看起来像这样
root@spring:repo# tree
.
└── docker
└── registry
└── v2
├── blobs
│ └── sha256
│ ├── 7b
│ └── ec
└── repositories
8 directories, 0 files
但是当我再次尝试推送已删除的图像时，我得到了
root@sea:scripts# docker push dockreg:5000/mubu4:v4
The push refers to repository [dockreg:5000/mubu4]
9f54eef41275: Layer already exists
v4: digest: sha256:7bd0d9a9821815dccb5c53c18cea04591ec633e2e529c5cdd39681169589c17f size: 529
在我的注册表中，我之前删除的 mubu4 图像文件夹已通过上述推送命令神秘地重新创建。
root@spring:repo# tree
.
└── docker
└── registry
└── v2
├── blobs
│ └── sha256
│ ├── 7b
│ └── ec
└── repositories
└── mubu4
└── _manifests
├── revisions
│ └── sha256
│ └── 7bd0d9a9821815dccb5c53c18cea04591ec633e2e529c5cdd39681169589c17f
│ └── link
└── tags
└── v4
├── current
│ └── link
└── index
└── sha256
└── 7bd0d9a9821815dccb5c53c18cea04591ec633e2e529c5cdd39681169589c17f
└── link
19 directories, 3 files
我还尝试用
root@spring:repo# rm -rf *
无济于事。之后尝试推送已删除的图像，仍然输出完全相同的Layer already exists错误，并且注册表树再次自动重新创建，看起来与上述树输出中的完全相同。
问题是我还能尝试什么来完成这项工作，和/或替代地，
从上面的测试可以看出，在 docker registry kubernetes pod 中，还有其他文件，这些文件保存了已删除图像似乎没有被删除的配置，并且这些文件通过 docker push 调用激活了已删除图像的重新创建。除了树我应该往哪里看
/var/lib/registry/docker/registry/v2/
所以我可以删除对已删除图像的所有引用？