VAST — Visibility Across Space and Time
VAST is a network telemetry engine for data-driven security investigations. It ingests high-volume streams of network and logs data, indexes it for later retrieval, and offers several ways to access the data efficiently. The goal is to make network forensics more productive and enable data scientists to tap into the rich world of network event data.
- Built for network forensics: VAST is purpose-built for SecOps (incident responders, threat hunters) at the intersection of data science. Security investigations are data investigations, and VAST puts the analyst back in charge, helping to get to the right subset of telemetry for the problem at hand.
- Interactive queries: VAST’s multi-level indexing delivers sub-second response times over the entire telemetry set—perfect for explorative threat-hunting workflows.
- High-throughput streaming: VAST relies on end-to-end streaming to ingest massive amounts of data. Dynamic backpressure ensures that the system does not keel over when stuffing too much data into it.
- Rich Data Model: VAST’s type-rich data model helps to retain domain semantics with a flexible schema and query language. All types support meaningful operations, e.g., IP address support top-k prefix search and containers membership queries. Moreover, VAST’s typed expression syntax allows you to search over fields having a particular type.
- Unfederated data access: VAST defines a portable framing for messages and files to enable access to the data from various platforms. The zero-copy export mechanism makes data sharing with downstream analytics applications incredibly efficient. This empowers data scientists to work on their analytics, as opposed to building tools for parsing and plumbing.
VAST stands for Visibility Across Space and Time to reflect the key benefit for users: make it easy to express temporal and spatial event relationships to illuminate your network analysis.
- The long option name
pcapparser is now called
--emit-file-headers(plural) to streamline it with the
nicloader and the new capability to process concatenated PCAP files. #3513
decapsulateoperator no longer drops the PCAP packet data in incoming events. #3515
- The new
s3connector enables the user to import/export file data from/to S3 buckets. #3496
- The new
zmqconnector ships with a saver and loader for interacting with ZeroMQ. The loader (source) implements a connecting
SUBsocket and the saver (sink) a binding
--connectflags make it possible to control the direction of connection establishment. #3497
- The new
gcsconnector enables the user to import/export file data from/to GCS buckets. #3498
- The new connectors
ftpssimplify using remote files in pipelines via HTTP(S) and FTP(S). #3499
- The new
linesparser splits its input at newline characters and produces events with a single field containing the line. #3511
pcapparser can now process a stream of concatenated PCAP files. On the command line, you can now parse traces with
cat *.pcap | tenzir 'read pcap'. When providing
--emit-file-headers, each intermediate file header yields a separate event. #3513
nicloader has a new option
--emit-file-headersthat prepends a PCAP file header for every batch of bytes that the loader produces, yielding a stream of concatenated PCAP files. #3513
- You can now write
show nicsto get a list of network interfaces. Use
show nics | select nameto a get a list of possible interface names for
from nic. #3517
- Pipelines now show up in the “stopped” instead of the “created” state after the node restarted. #3487
Copyright (c) 2014, Tenzir GmbH