k8s中pod的日誌收集有2種常見的解決方案；
方案一：使用fluentd作爲daemonset收集stdout和/var/lib/containers目錄下的所有日誌（因爲對fluentd不太熟悉，所以感覺麻煩）；
方案二：使用filebeat作爲sidecar方式（這種方式過於繁瑣，需要在每個pod中添加這個容器）
無意間發現阿里雲開源的log-pilot收集k8s的日誌真的超級方便，配置也簡單；
官方介紹：
github地址：https://github.com/AliyunContainerService/log-pilot
log-pilot官方介紹：https://yq.aliyun.com/articles/674327
log-pilot官方搭建：https://yq.aliyun.com/articles/674361?spm=a2c4e.11153940.0.0.21ae21c3mTKwWS
log-pilot的daemonset文件：
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: log-pilot
labels:
app: log-pilot
# 設置期望部署的namespace
namespace: kube-system
spec:
updateStrategy:
type: RollingUpdate
template:
metadata:
labels:
app: log-pilot
annotations:
scheduler.alpha.kubernetes.io/critical-pod: ''
spec:
# 是否允許部署到Master節點上
tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule
containers:
- name: log-pilot
# 版本請參考https://github.com/AliyunContainerService/log-pilot/releases
image: registry.cn-hangzhou.aliyuncs.com/acs/log-pilot:0.9.7-filebeat
resources:
limits:
memory: 500Mi
requests:
cpu: 200m
memory: 200Mi
env:
- name: "NODE_NAME"
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: "LOGGING_OUTPUT"
value: "elasticsearch"
# 請確保集羣到ES網絡可達
- name: "ELASTICSEARCH_HOSTS"
value: "10.10.5.78:9200"
# 配置ES訪問權限
#- name: "ELASTICSEARCH_USER"
# value: "{es_username}"
#- name: "ELASTICSEARCH_PASSWORD"
# value: "{es_password}"
volumeMounts:
- name: sock
mountPath: /var/run/docker.sock
- name: root
mountPath: /host
readOnly: true
- name: varlib
mountPath: /var/lib/filebeat
- name: varlog
mountPath: /var/log/filebeat
- name: localtime
mountPath: /etc/localtime
readOnly: true
livenessProbe:
failureThreshold: 3
exec:
command:
- /pilot/healthz
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 2
securityContext:
capabilities:
add:
- SYS_ADMIN
terminationGracePeriodSeconds: 30
volumes:
- name: sock
hostPath:
path: /var/run/docker.sock
- name: root
hostPath:
path: /
- name: varlib
hostPath:
path: /var/lib/filebeat
type: DirectoryOrCreate
- name: varlog
hostPath:
path: /var/log/filebeat
type: DirectoryOrCreate
- name: localtime
hostPath:
path: /etc/localtime創建nginx測試pod收集日誌示例：
apiVersion: apps/v1beta2 kind: Deployment metadata: name: node-affinity spec: selector: matchLabels: app: node-affinity replicas: 3 template: metadata: labels: app: node-affinity spec: containers: - name: nginx image: nginx imagePullPolicy: IfNotPresent env: - name: aliyun_logs_nginx value: "stdout" --- apiVersion: v1 kind: Service metadata: name: node-affinity spec: selector: app: node-affinity ports: - port: 80 targetPort: 80 type: NodePort
創建tomcat測試pod收集日誌示例：
apiVersion: v1
kind: Pod
metadata:
name: tomcat
spec:
containers:
- name: tomcat
image: "tomcat:8.0"
env:
# 1、stdout爲約定關鍵字，表示採集標準輸出日誌
# 2、配置標準輸出日誌採集到ES的catalina索引下
- name: aliyun_logs_catalina
value: "stdout"
# 1、配置採集容器內文件日誌，支持通配符
# 2、配置該日誌採集到ES的access索引下
- name: aliyun_logs_access
value: "/usr/local/tomcat/logs/catalina.*.log"
# 容器內文件日誌路徑需要配置emptyDir
volumeMounts:
- name: tomcat-log
mountPath: /usr/local/tomcat/logs
volumes:
- name: tomcat-log
emptyDir: {}