Package com.opengamma.strata.collect.io

Class CsvIterator

  • All Implemented Interfaces:
    PeekingIterator<CsvRow>, AutoCloseable, Iterator<CsvRow>
    public final class CsvIterator
extends Object
implements AutoCloseable, PeekingIterator<CsvRow>
    Iterator over the rows of a CSV file.

    Provides the ability to iterate over a CSV file together with the ability to parse it from a CharSource. The separator may be specified, allowing TSV files (tab-separated) and other similar formats to be parsed. See CsvFile for more details of the CSV format.

    This class processes the CSV file row-by-row. To load the entire CSV file into memory, use CsvFile.

    This class must be used in a try-with-resources block to ensure that the underlying CSV file is closed: 

      try (CsvIterator csvIterator = CsvIterator.of(source, true)) {
    // use the CsvIterator
  }
    One way to use the iterable is with the for-each loop, using asIterable(): 
      try (CsvIterator csvIterator = CsvIterator.of(source, true)) {
    for (CsvRow row : csvIterator.asIterable()) {
      // process the row
    }
  }
    This class also allows the headers to be obtained without reading the whole CSV file: 
      try (CsvIterator csvIterator = CsvIterator.of(source, true)) {
    ImmutableList<String> headers = csvIterator.headers();
  }

    • Method Detail

      • of

        public static CsvIterator of​(CharSource source,
                             boolean headerRow)
        Parses the specified source as a CSV file, using a comma as the separator.
        Parameters:
        source - the source to read as CSV
        headerRow - whether the source has a header row, an empty source must still contain the header
        Returns:
        the CSV file
        Throws:
        UncheckedIOException - if an IO exception occurs
        IllegalArgumentException - if the file cannot be parsed

      • of

        public static CsvIterator of​(CharSource source,
                             boolean headerRow,
                             char separator)
        Parses the specified source as a CSV file where the separator is specified and might not be a comma.

        This overload allows the separator to be controlled. For example, a tab-separated file is very similar to a CSV file, the only difference is the separator.

        Parameters:
        source - the source to read as CSV
        headerRow - whether the source has a header row, an empty source must still contain the header
        separator - the separator used to separate each field, typically a comma, but a tab is sometimes used
        Returns:
        the CSV file
        Throws:
        UncheckedIOException - if an IO exception occurs
        IllegalArgumentException - if the file cannot be parsed

      • of

        public static CsvIterator of​(Reader reader,
                             boolean headerRow)
        Parses the specified reader as a CSV file, using a comma as the separator.

        The caller is responsible for closing the reader, such as by calling close().

        Parameters:
        reader - the file reader
        headerRow - whether the source has a header row, an empty source must still contain the header
        Returns:
        the CSV file
        Throws:
        UncheckedIOException - if an IO exception occurs
        IllegalArgumentException - if the file cannot be parsed

      • of

        public static CsvIterator of​(Reader reader,
                             boolean headerRow,
                             char separator)
        Parses the specified reader as a CSV file where the separator is specified and might not be a comma.

        This overload allows the separator to be controlled. For example, a tab-separated file is very similar to a CSV file, the only difference is the separator.

        The caller is responsible for closing the reader, such as by calling close().

        Parameters:
        reader - the file reader
        headerRow - whether the source has a header row, an empty source must still contain the header
        separator - the separator used to separate each field, typically a comma, but a tab is sometimes used
        Returns:
        the CSV file
        Throws:
        UncheckedIOException - if an IO exception occurs
        IllegalArgumentException - if the file cannot be parsed

      • headers

        public ImmutableList<String> headers()
        Gets the header row.

        If there is no header row, an empty list is returned.

        Returns:
        the header row

      • containsHeader

        public boolean containsHeader​(String header)
        Checks if the header is present in the file.

        Matching is case insensitive.

        Parameters:
        header - the column header to match
        Returns:
        true if the header is present

      • containsHeaders

        public boolean containsHeaders​(Collection<String> headers)
        Checks if the headers are present in the file.

        Matching is case insensitive.

        Parameters:
        headers - the column headers to match
        Returns:
        true if all the headers are present

      • containsHeader

        public boolean containsHeader​(Pattern headerPattern)
        Checks if the header pattern is present in the file.

        Matching is case insensitive.

        Parameters:
        headerPattern - the header pattern to match
        Returns:
        true if the header is present

      • asIterable

        public Iterable<CsvRow> asIterable()
        Returns an Iterable that wraps this iterator.

        Unlike most Iterable implementations, the method Iterable.iterator() can only be called once. This is intended for use with the for-each loop. 

          try (CsvIterator csvIterator = CsvIterator.of(source, true)) {
    for (CsvRow row : csvIterator.asIterable()) {
      // process the row
    }
  }
        Returns:
        this iterator as an Iterable

      • asStream

        public Stream<CsvRow> asStream()
        Returns a stream that wraps this iterator.

        The stream will process any remaining rows in the CSV file. As such, it is recommended that callers should use this method or the iterator methods and not both.

        Returns:
        the stream wrapping this iterator

      • nextBatch

        public List<CsvRow> nextBatch​(int count)
        Returns the next batch of rows from the CSV file.

        This will return up to the specified number of rows from the file at the current iteration point. An empty list is returned if there are no more rows.

        Parameters:
        count - the number of rows to try and get, negative returns an empty list
        Returns:
        the next batch of rows, up to the number requested
        Throws:
        UncheckedIOException - if an IO exception occurs
        IllegalArgumentException - if the file cannot be parsed

      • nextBatch

        public List<CsvRow> nextBatch​(Predicate<CsvRow> selector)
        Returns the next batch of rows from the CSV file using a predicate to determine the rows.

        This is useful for CSV files where information is grouped with an identifier or key. For example, a variable notional trade file might have one row for the trade followed by multiple rows for the variable aspects, all grouped by a common trade identifier. In general, callers should peek or read the first row and use information within it to create the selector: 

          while (it.hasNext()) {
    CsvRow first = it.peek();
    String id = first.getValue("ID");
    List<CsvRow> batch = it.nextBatch(row -> row.getValue("ID").equals(id));
    // process batch
  }
        This will return a batch of rows where the selector returns true for the row. An empty list is returned if the selector returns false for the first row.
        Parameters:
        selector - selects whether a row is part of the batch or part of the next batch
        Returns:
        the next batch of rows, as determined by the selector
        Throws:
        UncheckedIOException - if an IO exception occurs
        IllegalArgumentException - if the file cannot be parsed

      • toString

        public String toString()
        Returns a string describing the CSV iterator.
        Overrides:
        toString in class Object
        Returns:
        the descriptive string