Question

我想知道如何有效地删除特定文件的前N行。

当然，我们可以将完整的文件加载到内存中，然后删除前N行并将完整的更改内容写入文件。

但是，我想知道是否有办法比仅将所有内容加载到内存中更有效。

Answer 1

文件系统不支持从文件的任意位置删除，因此您至少需要自己重写整个文件。实施所需内容的常用方法是

打开一个临时文件，
从原始文件中读取小片段，例如在您的情况下，每次只能一行一行
仅将要保留在原始文件中的行写到临时文件中
完成后将临时文件重命名为原始文件名。

Answer 2

您是否要尝试执行类似于记录截断和旋转的操作？

一种常见的模式可能会帮助您：将日志拆分为多个文件，并按照创建顺序覆盖或删除文件。

例如，您可能拥有2018-09-09.log.txt或log.1.txt并每小时旋转一次到新的日志文件。这样可以删除较旧的文件以释放空间。

除此之外，我认为没有一种特定的方法可以在不重写文件其余部分的情况下在Go中删除文件的早期行。

Answer 3

最有效的方法是向后读取文件。

/*
    This package implements funcationality similar to the UNIX command tail.
    It prints out the last "x" many lines of a file, usually 10.
    The number of lines in a file is determined by counting newline (\n) characters.
    This can be use either from CLI or imported into other projects.
*/

package main

import (
    "errors"
    "flag"
    "fmt"
    "io"
    "os"
)

var (
    //DEFINE FLAG DEFAULTS
    filename = ""
    numLines = 10

    //ErrNoFilename is thrown when the path the the file to tail was not given
    ErrNoFilename = errors.New("You must provide the path to a file in the \"-file\" flag.")

    //ErrInvalidLineCount is thrown when the user provided 0 (zero) as the value for number of lines to tail
    ErrInvalidLineCount = errors.New("You cannot tail zero lines.")
)

//GET FLAGS FROM CLI
func init() {
    flag.StringVar(&filename, "file", "", "Filename to get last lines of text from.")
    flag.IntVar(&numLines, "n", numLines, "Number of lines to get from end of file.")
    flag.Parse()
}

//MAIN FUNCTIONALITY OF APP
//make sure filename (path to file) was given
//run it through the tailing function
//print the output to stdout
func main() {
    //TAIL
    text, err := GoTail(filename, numLines)
    if err != nil {
        fmt.Println(err)
        return
    }

    //DONE
    fmt.Print(text)
    return
}

//GoTail IS THE FUNCTION THAT ACTUALLY DOES THE "TAILING"
//this can be used this package is imported into other golang projects
func GoTail(filename string, numLines int) (string, error) {
    //MAKE SURE FILENAME IS GIVEN
    //actually, a path to the file
    if len(filename) == 0 {
        return "", ErrNoFilename
    }

    //MAKE SURE USER WANTS TO GET AT LEAST ONE LINE
    if numLines == 0 {
        return "", ErrInvalidLineCount
    }

    //OPEN FILE
    file, err := os.Open(filename)
    if err != nil {
        return "", err
    }
    defer file.Close()

    //SEEK BACKWARD CHARACTER BY CHARACTER ADDING UP NEW LINES
    //offset must start at "-1" otherwise we are already at the EOF
    //"-1" from numLines since we ignore "last" newline in a file
    numNewLines := 0
    var offset int64 = -1
    var finalReadStartPos int64
    for numNewLines <= numLines-1 {
        //seek to new position in file
        startPos, err := file.Seek(offset, 2)
        if err != nil {
            return "", err
        }

        //make sure start position can never be less than 0
        //aka, you cannot read from before the file starts
        if startPos == 0 {
            //set to -1 since we +1 to this below
            //the position will then start from the first character
            finalReadStartPos = -1
            break
        }

        //read the character at this position
        b := make([]byte, 1)
        _, err = file.ReadAt(b, startPos)
        if err != nil {
            return "", err
        }

        //ignore if first character being read is a newline
        if offset == int64(-1) && string(b) == "\n" {
            offset--
            continue
        }

        //if the character is a newline
        //add this to the number of lines read
        //and remember position in case we have reached our target number of lines
        if string(b) == "\n" {
            numNewLines++
            finalReadStartPos = startPos
        }

        //decrease offset for reading next character
        //remember, we are reading backward!
        offset--
    }

    //READ TO END OF FILE
    //add "1" here to move offset from the newline position to first character in line of text
    //this position should be the first character in the "first" line of data we want
    endPos, err := file.Seek(int64(-1), 2)
    if err != nil {
        return "", err
    }
    b := make([]byte, (endPos+1)-finalReadStartPos)
    _, err = file.ReadAt(b, finalReadStartPos+1)
    if err == io.EOF {
        return string(b), nil
    } else if err != nil {
        return "", err
    }

    //special case
    //if text is read, then err == io.EOF should hit
    //there should *never* not be an error above
    //so this line should never return
    return "**No error but no text read.**", nil
}

在控制台中，您必须输入：gotail -file <filename> -n=<number of lines> 因此，您可以读取文件末尾的最后几行。如果您想要除前10行之外的所有行，那么我将更改此代码。

Answer 4

您可以使用wc -l <fileName>查找文件中的总行数，将其设为x，并假设您要读取除前1000条之外的所有行，然后使用tail -n<x-1000> <fileName>。

一种有效的方法是使用sed。参见this答案

删除文件的前N行

4 个答案: