Seafile K8S advanced management

This document mainly describes how to manage and maintain Seafile deployed through our K8S deployment document. At the same time, if you are already proficient in using kubectl commands to manage K8S resources, you can also customize the deployment solutions we provide.

Namespaces for Seafile K8S deployment

Our documentation provides two deployment solutions for both single-node and cluster deployment (via Seafile Helm Chart and K8S resource files), both of which can be highly customized.

Regardless of which deployment method you use, in our newer manuals (usually in versions after Seafile 12.0.9), Seafile-related K8S resources (including related Pods, services, and persistent volumes, etc.) are defined in the seafile namespace. In previous versions, you may deploy Seafile in the default namespace, so in this case, when referring to this document for Seafile K8S resource management, be sure to remove -n seafile in the command.

Helm Chart users can configure a standard Ingress or Gateway API HTTPRoute in my-values.yaml. See the single-node Helm guide or cluster Helm guide. The manual Gateway and HTTPRoute resources below are intended for K8S resource-file deployments or advanced Helm routing requirements.

Seafile K8S Container management

Similar to docker installation, you can also manage containers through some kubectl commands. For example, you can use the following command to check whether the relevant resources are started successfully and whether the relevant services can be accessed normally. First, execute the following command and remember the pod name with seafile- as the prefix (such as seafile-748b695648-d6l4g)

kubectl get pods -n seafile

You can check a status of a pod by

kubectl logs seafile-748b695648-d6l4g -n seafile

and enter a container by

kubectl exec -it seafile-748b695648-d6l4g -n seafile --  bash

Also, you can restart the services by the following commands:

kubectl delete pods -n seafile $(kubectl get pods -n seafile -o jsonpath='{.items[*].metadata.name}' | grep seafile)

K8S Gateway and HTTPS

The Kubernetes Ingress API is frozen and Gateway API is its modern successor. Both can expose Seafile, depending on the networking controller available in your cluster.

Helm Chart routing

For Helm Chart 13.0.4 and later, configure one of the following options in my-values.yaml and run helm upgrade. Select one method unless you intentionally need two routes.

IngressGateway API HTTPRoute

Use this option when an Ingress controller is installed:

ingress:
  enabled: true
  className: nginx
  annotations: {}
  hosts:
    - host: seafile.example.com
      paths:
        - path: /
          pathType: Prefix
  tls:
    - secretName: seafile-tls-cert
      hosts:
        - seafile.example.com

The chart creates an Ingress named seafile, forwarding the configured paths to the seafile Service on port 80. The TLS Secret must be created separately.

Use this option when Gateway API v1 CRDs and a Gateway controller are installed. The Gateway must already exist and permit routes from the Seafile namespace:

httpRoute:
  enabled: true
  annotations: {}
  parentRefs:
    - name: seafile-gateway
      # namespace: gateway-namespace  # Required when the Gateway is in another namespace.
  hostnames:
    - seafile.example.com

The chart creates an HTTPRoute named seafile, with a fixed / path-prefix match forwarding to seafile:80. Gateway listeners, TLS certificates, and custom routing rules remain the responsibility of the Gateway administrator.

For commands and edition-specific deployment details, see the single-node Helm guide or cluster Helm guide.

Manual Gateway API configuration

Use the following instructions when deploying with K8S resource files or when the chart-managed route does not meet your routing requirements.

For the details and features about K8S Gateway, please refer to the K8S official document, you can simpily install it by

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.1.0/standard-install.yaml

The Gateway API requires configuration of three API categories in its resource model: - GatewayClass: Defines a group of gateways with the same configuration, managed by the controller that implements the class. - Gateway: Defines an instance of traffic handling infrastructure, which can be thought of as a load balancer. - HTTPRoute: Defines HTTP-specific rules for mapping traffic from gateway listeners to representations of backend network endpoints. These endpoints are typically represented as Services.

GatewayClass

The GatewayClass resource serves the same purpose as the IngressClass in the old-ingress API, similar to the StorageClass in the Storage API. It defines the categories of Gateways that can be created. Typically, this resource is provided by your infrastructure platform, such as EKS or GKE. It can also be provided by a third-party Ingress Controller, such as Nginx-gateway or Istio-gateway.

Here, we take the Nginx-gateway for the example, and you can install it with the official document. After installation, you can view the installation status with the following command:

# `gc` means the `gatewayclass`, and its same as `kubectl get gatewayclass`
kubectl get gc 

#NAME    CONTROLLER                                   ACCEPTED   AGE
#nginx   gateway.nginx.org/nginx-gateway-controller   True       22s

Typically, after you install GatewayClass, your cloud provider will provide you with a load balancing IP, which is visible in GatewayClass. If this IP is not assigned, you can manually bind it to a IP that can be accessed from exteranl network.

kubectl edit svc nginx-gateway -n nginx-gateway

and modify the following section:

...
spec:
    ...
    externalIPs:
    - <your external IP>
    externalTrafficPolicy: Cluster
    ...
...

Gateway

Gateway is used to describe an instance of traffic processing infrastructure. Usually, Gateway defines a network endpoint that can be used to process traffic, that is, to filter, balance, split, etc. Service and other backends. For example, it can represent a cloud load balancer, or a cluster proxy server configured to accept HTTP traffic. As above, please refer to the official documentation for a detailed description of Gateway. Here is only a simple reference configuration for Seafile:

# nano seafile-gateway/gateway.yaml

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: seafile-gateway
spec:
  gatewayClassName: nginx
  listeners:
  - name: seafile-http
    protocol: HTTP
    port: 80

HTTPRoute

The HTTPRoute category specifies the routing behavior of HTTP requests from the Gateway listener to the backend network endpoints. For service backends, the implementation can represent the backend network endpoint as a service IP or a supporting endpoint of the service. It represents the configuration applied to the underlying Gateway implementation. For example, defining a new HTTPRoute may result in configuring additional traffic routes in a cloud load balancer or in-cluster proxy server. As above, please refer to the official documentation for a detailed description of the HTTPRoute resource. This manual example is for independently managed routing only; do not apply it together with the chart-managed httpRoute, because both use the seafile backend and may create duplicate routes.

# nano seafile-gateway/httproute.yaml

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: seafile-httproute
spec:
  parentRefs:
  - group: gateway.networking.k8s.io
    kind: Gateway
    name: seafile-gateway
  hostnames:
  - "<your domain>"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: seafile
      port: 80

After installing or defining GatewayClass, Gateway and HTTPRoute, you can now enable this feature by following command and view your Seafile server by the URL http://seafile.example.com/:

kubectl apply -f seafile-gateway -n seafile

Enable HTTPS (Optional)

When using K8S Gateway, a common way to enable HTTPS is to add relevant information about the TLS listener in Gateway resource. You can refer here for futher details. We will provide a simple way here so that you can quickly enable HTTPS for your Seafile K8S.

1. Create a secret resource (seafile-tls-cert) for your TLS certificates:

   kubectl create secret tls seafile-tls-cert \
   --cert=<your path to fullchain.pem> \
   --key=<your path to privkey.pem>
   

   2. Use the TLS in your Gateway resource and enable HTTPS:

   # nano seafile-gateway/gateway.yaml
   
   ...
   spec:
     ...
     listeners:
       - name: seafile-http
         ...
         tls:
           certificateRefs:
           - kind: Secret
               group: ""
               name: seafile-tls-cert
   ...
   

2. Modify seahub_settings.py:

   SERVICE_URL = "https://<your domain>/"
   

3. Restart Seafile K8S Gateway:

   kubectl delete -f seafile-gateway -n seafile
   kubectl apply -f seafile-gateway -n seafile
   

   Now you can access your Seafile service in https://<your domain>/

Log routing and aggregating system

Similar to single-node deployment, you can browse the log files of Seafile running directly in the persistent volume directory (i.e., <path>/seafile/logs). The difference is that when using K8S to deploy a Seafile cluster (especially in a cloud environment), the persistent volume created is usually shared and synchronized for all nodes. However, the logs generated by the Seafile service do not record the specific node information where these logs are located, so browsing the files in the above folder may make it difficult to identify which node these logs are generated from. Therefore, one solution proposed here is:

1. Record the generated logs to the standard output. In this way, the logs can be distinguished under each node by kubectl logs (but all types of logs will be output together now). For K8S resource-file deployments, enable this feature by modifying SEAFILE_LOG_TO_STDOUT to true in seafile-env.yaml:

   ...
   data:
     ...
     SEAFILE_LOG_TO_STDOUT: "true"
     ...
   

   Current Helm Chart 13.0 values enable SEAFILE_LOG_TO_STDOUT: "true" by default for single-node and cluster deployments. Helm users should change seafile.env.SEAFILE_LOG_TO_STDOUT in my-values.yaml and run helm upgrade rather than edit seafile-env.yaml.

   Then restart the Seafile server:

   kubectl delete pods -n seafile $(kubectl get pods -n seafile -o jsonpath='{.items[*].metadata.name}' | grep seafile)
   

2. Since the logs in step 1 can be distinguished between nodes, but they are aggregated and output together, it is not convenient for log retrieval. So you have to route the standard output logs (i.e., distinguish logs by corresponding components name) and re-record them in a new file or upload them to a log aggregation system (e.g., Loki).

Currently in the K8S environment, the commonly used log routing plugins are:

• Fluent Bit
• Fluentd
• Logstash
• Promtail (also a part of Loki)

Fluent Bit and Promtail are more lightweight (i.e., consume less system resources), while Promtail only supports transferring logs to Loki. Therefore, this document will mainly introduce log routing through Fluent Bit which is a fast, lightweight logs and metrics agent. It is also a CNCF graduated sub-project under the umbrella of Fluentd. Fluent Bit is licensed under the terms of the Apache License v2.0. You should deploy the Fluent Bit in your K8S cluster by following offical document firstly. Then modify Fluent-Bit pod settings to mount a new directory to load the configuration files:

#kubectl edit ds fluent-bit

...
spec:
    ...
    spec:
        ...
        containers:
        - name: fluent-bit
          volumeMounts:
            ...
            - mountPath: /fluent-bit/etc/seafile
              name: fluent-bit-seafile
            - mountPath: /
        ...
    ...
    volumes:
    ...
    - hostPath:
        path: /opt/fluent-bit
      name: fluent-bit-seafile

and

#kubectl edit cm fluent-bit

data:
    ...
    fluent-bit.conf: |
        [SERVICE]
            ...
            Parsers_File /fluent-bit/etc/seafile/confs/parsers.conf
        ...
        @INCLUDE /fluent-bit/etc/seafile/confs/*-log.conf

For example in here, we use /opt/fluent-bit/confs (it has to be non-shared). What's more, the parsers will be defined in /opt/fluent-bit/confs/parsers.conf, and for each type log (e.g., seahub's log, seafevent's log) will be defined in /opt/fluent-bit/confs/*-log.conf. Each .conf file defines several Fluent-Bit data pipeline components:

Pipeline	Description	Required/Optional
INPUT	Specifies where and how Fluent-Bit can get the original log information, and assigns a tag for each log record after read.	Required
PARSER	Parse the read log records. For K8S Docker runtime logs, they are usually in Json format.	Required
FILTER	Filters and selects log records with a specified tag, and assigns a new tag to new records.	Optional
OUTPUT	tells Fluent-Bit what format the log records for the specified tag will be in and where to output them (such as file, Elasticsearch, Loki, etc.).	Required

Warning

For PARSER, it can only be stored in /opt/fluent-bit/confs/parsers.conf, otherwise the Fluent-Bit cannot startup normally.

Inputer

According to the above, a container will generate a log file (usually in /var/log/containers/<container-name>-xxxxxx.log), so you need to prepare an importer and add the following information (for more details, please refer to offical document about TAIL inputer) in /opt/fluent-bit/confs/seafile-log.conf:

[INPUT]
    Name              tail
    Path              /var/log/containers/seafile-frontend-*.log
    Buffer_Chunk_Size 2MB
    Buffer_Max_Size   10MB
    Docker_Mode       On
    Docker_Mode_Flush 5
    Tag               seafile.*
    Parser            Docker # for definition, please see the next section as well

[INPUT]
    Name              tail
    Path              /var/log/containers/seafile-backend-*.log
    Buffer_Chunk_Size 2MB
    Buffer_Max_Size   10MB
    Docker_Mode       On
    Docker_Mode_Flush 5
    Tag               seafile.*
    Parser            Docker

The above defines two importers, which are used to monitor seafile-frontend and seafile-backend services respectively. The reason why they are written together here is that for a node, you may not know when it will run the frontend service and when it will run the backend service, but they have the same tag prefix seafile..

Parser

Each input has to use a parser to parse the logs and pass them to the filter. Here, a parser named Docker is created to parse the logs generated by the K8S-docker-runtime container. The parser is placed in /opt/fluent-bit/confs/parser.conf (for more details, please refer to offical document about JSON parser):

[PARSER]
    Name        Docker
    Format      json
    Time_Key    time
    Time_Format %Y-%m-%dT%H:%M:%S.%LZ

Log records after parsing

The logs of the Docker container are saved in /var/log/containers in Json format (see the sample below), which is why we use the Json format in the above parser.

{"log":"[seaf-server] [2025-01-17 07:43:48] [INFO] seafile-session.c(86): fileserver: web_token_expire_time = 3600\n","stream":"stdout","time":"2025-01-17T07:43:48.294638442Z"}
{"log":"[seaf-server] [2025-01-17 07:43:48] [INFO] seafile-session.c(98): fileserver: max_index_processing_threads= 3\n","stream":"stdout","time":"2025-01-17T07:43:48.294810145Z"}
{"log":"[seaf-server] [2025-01-17 07:43:48] [INFO] seafile-session.c(111): fileserver: fixed_block_size = 8388608\n","stream":"stdout","time":"2025-01-17T07:43:48.294879777Z"}
{"log":"[seaf-server] [2025-01-17 07:43:48] [INFO] seafile-session.c(123): fileserver: max_indexing_threads = 1\n","stream":"stdout","time":"2025-01-17T07:43:48.295002479Z"}
{"log":"[seaf-server] [2025-01-17 07:43:48] [INFO] seafile-session.c(138): fileserver: put_head_commit_request_timeout = 10\n","stream":"stdout","time":"2025-01-17T07:43:48.295082733Z"}
{"log":"[seaf-server] [2025-01-17 07:43:48] [INFO] seafile-session.c(150): fileserver: skip_block_hash = 0\n","stream":"stdout","time":"2025-01-17T07:43:48.295195843Z"}
{"log":"[seaf-server] [2025-01-17 07:43:48] [INFO] ../common/seaf-utils.c(553): Use database Mysql\n","stream":"stdout","time":"2025-01-17T07:43:48.29704895Z"}

When these logs are obtained by the importer and parsed by the parser, they will become independent log records with the following fields:

• log: The original log content (i.e., same as you seen in kubectl logs seafile-xxx -n seafile) and an extra line break at the end (i.e., \n). This is also the field we need to save or upload to the log aggregation system in the end.
• stream: The original log come from. stdout means the standard output.
• time: The time when the log is recorded in the corresponding stream (ISO 8601 format).

Filter

Add two filters in /opt/fluent-bit/confs/seafile-log.conf for records filtering and routing. Here, the record_modifier filter is to select useful keys (see the contents in above tip label, only the log field is what we need) in the log records and rewrite_tag filter is used to route logs according to specific rules:

[FILTER]        
    Name record_modifier
    Match seafile.*
    Allowlist_key log


[FILTER]
    Name        rewrite_tag
    Match       seafile.*
    Rule        $log ^.*\[seaf-server\].*$ seaf-server false # for seafile's logs
    Rule        $log ^.*\[seahub\].*$ seahub false # for seahub's logs
    Rule        $log ^.*\[seafevents\].*$ seafevents false # for seafevents' lgos
    Rule        $log ^.*\[seafile-slow-rpc\].*$ seafile-slow-rpc false # for slow-rpc's logs

Output log's to Loki

Loki is multi-tenant log aggregation system inspired by Prometheus. It is designed to be very cost effective and easy to operate. The Fluent-Bit loki built-in output plugin allows you to send your log or events to a Loki service. It supports data enrichment with Kubernetes labels, custom label keys and Tenant ID within others.

Alternative Fluent-Bit Loki plugin by Grafana

For sending logs to Loki, there are two plugins for Fluent-Bit:

• The built-in Loki plugin maintained by the Fluent-Bit officially, and we will use it in this part because it provides the most complete features.
• Grafana-loki plugin maintained by Grafana Labs.

Due to each outputer dose not have a distinguishing marks in the configuration files (because Fluent-Bit takes each plugin as a tag workflow):

• Seaf-server log: Add an outputer to /opt/fluent-bit/confs/seaf-server-log.conf:

  [OUTPUT]
      Name        loki
      Match       seaf-server
      Host        <your Loki's host>
      port        <your Loki's port>
      labels      job=fluentbit, node_name=<your-node-name>, node_id=<your-node-id> # node_name and node_id is optional, but recommended for identifying the source node
  

• seahub log: Add an outputer to /opt/fluent-bit/confs/seahub-log.conf:

  [OUTPUT]
      Name        loki
      Match       seahub
      Host        <your Loki's host>
      port        <your Loki's port>
      labels      job=fluentbit, node_name=<your-node-name>, node_id=<your-node-id> # node_name and node_id is optional, but recommended for identifying the source node
  

• seafevents log: Add an outputer to /opt/fluent-bit/confs/seafevents-log.conf:

  [OUTPUT]
      Name        loki
      Match       seafevents
      Host        <your Loki's host>
      port        <your Loki's port>
      labels      job=fluentbit, node_name=<your-node-name>, node_id=<your-node-id> # node_name and node_id is optional, but recommended for identifying the source node
  

• seafile-slow-rpc log: Add an outputer to /opt/fluent-bit/confs/seafile-slow-rpc-log.conf:

  [OUTPUT]
      Name        loki
      Match       seafile-slow-rpc
      Host        <your Loki's host>
      port        <your Loki's port>
      labels      job=fluentbit, node_name=<your-node-name>, node_id=<your-node-id> # node_name and node_id is optional, but recommended for identifying the source node
  

Cloud Loki instance

If you are using a cloud Loki instance, you can follow the Fluent-Bit Loki plugin document to fill up all necessary fields. Usually, the following fields are additional needs in cloud Loki service:

• tls
• tls.verify
• http_user
• http_passwd