Class WarmerListStore

java.lang.Object
org.frontcache.warmer.WarmerListStore
public class WarmerListStore extends Object
The URI lists under FRONTCACHE_HOME/warmer/, and the only place a list name becomes a File.

Names are caller-supplied, so they are constrained

ConfigFile is an enum because the set of config files is closed. The set of lists is not - an operator names them - so the defence here is a check, written to be hard to get slightly wrong:
  • the name must match NAME. That character class cannot express a separator, a .. segment is impossible without a /, and a leading dot is excluded - which is also what keeps .state/ and this class's own temp files out of every listing;
  • the resolved file's canonical parent must be the canonical warmer/ directory, which covers a symlink placed in the directory by hand.

What a list's content is, byte for byte

UTF-8, LF line endings, no BOM. A write normalizes CRLF (and a lone CR) to LF and strips a BOM, and adds no trailing newline - saving a list unchanged must not change it. hash(String) is the SHA-256 of the content after that same normalization, so a file copied in by hand with CRLF endings hashes the same as the text the console shows for it.

  • Field Details

    • NAME

      public static final Pattern NAME
      See the class comment. 1 + 99 characters, then .txt.

  • Constructor Details

    • WarmerListStore

      public WarmerListStore(File dir)

  • Method Details

    • forNode

      public static WarmerListStore forNode()
      The store for this node: FRONTCACHE_HOME/warmer, created if absent. The directory is the one dump-keys has always written to.

    • getDirectory

      public File getDirectory()

    • nameProblem

      public static String nameProblem(String name)
      Returns:
      null when name is a valid list name, otherwise why it is not

    • resolve

      public File resolve(String name)
      Returns:
      the list's file, which need not exist
      Throws:
      IllegalArgumentException - when the name is not a list name or resolves outside warmer/

    • exists

      public boolean exists(String name)

    • failedListName

      public static String failedListName(String listName, String stamp)
      The name the failed entries of a run over listName go to. A re-run of a failed list does not stack suffixes - top.failed-A.txt fails into top.failed-B.txt, not top.failed-A.failed-B.txt - so the name stays inside NAME.

    • list

      public List<WarmerListInfo> list(long editMaxBytes) throws IOException
      Every list, newest first. Entry counts are cached per (name, mtime, size).
      Throws:
      IOException

    • info

      public WarmerListInfo info(String name, long editMaxBytes) throws IOException
      Throws:
      IOException

    • openReader

      public BufferedReader openReader(String name) throws IOException
      A UTF-8 reader over the list. readLine() already treats CRLF and a lone CR as LF.
      Throws:
      IOException

    • openRaw

      public InputStream openRaw(String name) throws IOException
      The raw bytes, for a download.
      Throws:
      IOException

    • read

      public WarmerListView read(String name, long editMaxBytes, int previewLines) throws IOException
      What get-warmer-list shows: the whole normalized content and its hash when the list is small enough to edit, otherwise the first and last previewLines lines.
      Returns:
      null when the list does not exist
      Throws:
      IOException

    • grep

      public List<WarmerLine> grep(String name, String needle, int max) throws IOException
      Lines containing needle (case-insensitive), with their line numbers - a streamed scan, so a 50k-line list can be searched without being loaded.
      Throws:
      IOException

    • normalize

      public static String normalize(String content)
      CRLF and a lone CR to LF, a leading BOM removed. Nothing else - no trailing newline added.

    • hashOf

      public static String hashOf(String content)
      SHA-256 (lowercase hex) of normalized text - the baseHash the console sends back.

    • hash

      public String hash(String name) throws IOException
      SHA-256 of the list's normalized content, streamed. Null when the list does not exist.
      Throws:
      IOException

    • cachedHash

      public String cachedHash(String name) throws IOException
      hash(String), remembered per (mtime, size). For the callers that ask repeatedly about the same file - the status poll's "is the checkpoint still resumable?" - and must not re-read 64 MB to find out.
      Throws:
      IOException

    • write

      public WarmerListInfo write(String name, InputStream body, long maxBytes, String baseHash, long editMaxBytes) throws IOException, WarmerListStore.RefusedException
      Creates or replaces one list from a stream, never holding it whole. The body goes into a temp file in warmer/, normalized as it is copied, and is moved into place atomically once it is complete - the ConfigFileStore mechanics, including the reason for the same directory: ATOMIC_MOVE across filesystems throws. Past maxBytes the copy stops, the temp file is deleted, and nothing is replaced.
      Parameters:
      baseHash - the hash the caller last read, or null to skip the concurrency check
      Returns:
      the list as written
      Throws:
      IOException
      WarmerListStore.RefusedException

    • delete

      public boolean delete(String name) throws IOException
      Removes one list. @return false when there was nothing to remove
      Throws:
      IOException

    • createFailedList

      public BufferedWriter createFailedList(String name) throws IOException
      A new file for a run's failed entries. CREATE_NEW: a name that already exists is never appended to or overwritten.
      Throws:
      IOException

    • checkpointFile

      public File checkpointFile()
      warmer/.state/run.json - a fixed name, never caller-supplied.