How to solve jenkins 'Disk space is too low' issue?

前端 未结 8 1256
南笙
南笙 2021-01-31 07:40

I have deployed Jenkins in my CentOS machine, Jenkins was working well for 3 days, but yesterday there was a Disk space is too low. Only 1.019GB left.

相关标签:
8条回答
  • 2021-01-31 08:21

    You can easily change the threshold from jenkins UI (my version is 1.651.3):

    [jenkins nodes preventing monitoring page]


    Update: How to ensure high disk space

    This feature is meant to prevent working on slaves with low free disk space. Lowering the threshold would not solve the fact that some jobs do not properly cleanup after they finish.

    Depending on what you're building:

    1. Make sure you understand what is the disk output of your build - if possible - restrict the output to happen only to the job workspace. Use workspace cleanup plugin to cleanup the workspace as post build step.
    2. If the process must write some data to external folders - clean them up manually on post build steps.

    Alternative1 - provision a new slave per job (use spot slaves - there are many plugins that integrate with different cloud provider to provision on the fly machines on demand)

    Alternative2 - run the build inside a container. Everything will be discarded once the build is finished

    0 讨论(0)
  • 2021-01-31 08:22

    I have a cleanup job with the following build steps. You can schedule it @daily or @weekly.

    1. Execute system groovy script build step to clean up old jobs:
        import jenkins.model.Jenkins
        import hudson.model.Job
    
        BUILDS_TO_KEEP = 5
    
        for (job in Jenkins.instance.items) {
          println job.name
    
          def recent = job.builds.limit(BUILDS_TO_KEEP)
    
          for (build in job.builds) {
            if (!recent.contains(build)) {
              println "Preparing to delete: " + build
              build.delete()
            }
          }
        }
    

    You'd need to have Groovy plugin installed.

    1. Execute shell build step to clean cache directories
    rm -r ~/.gradle/
    rm -r ~/.m2/
    
    echo "Disk space"
    du -h -s /
    
    0 讨论(0)
  • 2021-01-31 08:28

    For people who do not know where the configs are, download the tmpcleaner from https://updates.jenkins-ci.org/download/plugins/tmpcleaner/

    You will get an hpi file here. Go to Manage Jenkins-> Manage plugins-> Advanced and then upload the hpi file here and restart jenkins

    You can immediately see a difference if you go to Manage Nodes.

    Since my jenkins was installed in a debian server, I did not understand most of the answers related to this since i cannot find a /etc/default folder or jenkins file. If someone knows where the /tmp folder is or how to configure it for debian , do let me know in comments

    0 讨论(0)
  • 2021-01-31 08:31

    I got the same issue. My jenkins version is 2.3 and its UI is slightly different. Putting it here so that it may helps someone. Increasing both disk space thresholds to 5GB fixed the issue.

    0 讨论(0)
  • 2021-01-31 08:35

    You can limit the reduce of disc space by discarding the old builds. There's a checkbox for this in the project configuration.

    0 讨论(0)
  • 2021-01-31 08:35

    This is actually a legitimate question so I don't understand the downvotes, perhaps it belongs on Superuser or Serverfault. This is a soft warning threshold not hard limit where the disk is out of space.

    • For hudson see where to configure hudson node disk temp space thresholds - this is talking about the host, not nodes

    Jenkins is the same. The conclusion is for many small projects the system property called hudson.diagnosis.HudsonHomeDiskUsageChecker.freeSpaceThreshold could be decreased.

    In saying that I haven't tested it and there is a disclaimer

    No compatibility guarantee

    In general, these switches are often experimental in nature, and subject to change without notice. If you find some of those useful, please file a ticket to promote it to the official feature.

    0 讨论(0)
提交回复
热议问题