Field reference
This section describes how to use fields including where flexibility exists and the implications of that flexibility.
Schema and field references
Queries are evaluated against the events datastore. Every event type has a set of properties – we refer to them as fields – that carry data of a defined primitive type. For instance, every event has a sensor_id property that is of type string and a timestamp property of type timestamp. The full schema for all available event types and their properties is available within the Event Types page.
All queries consist fundamentally of matching an event field against a value; for instance, "Show me all events for which the destination IP is 8.8.8.8." However, there is some room for flexibility. Do you really want all event types, or is there one in particular you're interested. Do you really want to restrict results to cases where 8.8.8.8 is the destination IP address, or would any involvement of that IP address be interesting?
Each field involved in a query must be resolved to a specific field of a specific event type. A fully-specified field is of the format event-type:field; for instance, flow:sensor_id and dns:dst.geo.country are both fully specified. For a field that's not fully specified, either by omitting the event type or part of the field, the system will expand the field to include all fully-qualified fields that fit the ambiguity.
The next two subsections will show how these expansions work and what their implications are.
Event-type expansion
A field without a specifed event type will infer all valid event types. For example, dns and flow events both have a proto field, so a query containing just proto without an event-type prefix will expand to include both event types. Effectively, the query on the first line below is rewritten by the query engine on the backend to the query on the second line.
// original query
proto = 'udp'
// rewrite produced by the query engine on the backend
dns:proto = 'udp' OR flow:proto = 'udp'
If a field only belongs to one event type, then the event type does not need to be specified since the results would be the same. For example, the qtype_name field is unique to the dns event type, so only one event type can be inferred. This means that the two queries below are equivalent.
// original query
qtype_name = 'A'
// the rewrite is equivalent
dns:qtype_name = 'A'
Field expansion
Some fields hold values of a structural type (Event Type and Fields), meaning they contain subfields that must be referenced. To make this clear, let's use the src field as an example. The src field is of the type ip-object, i.e. a JSON structure. Looking at the following code block, we couldn't compare src to an IP address because we'd have to specify the entire JSON structure for them to match on structure. Instead, we must compare the ip subfield to an IP address.
// invalid because src is type ip-object and we're comparing it to an ip
src = 10.0.0.10
// valid because src.ip is type ip and we're comparing it to an ip
src.ip = 10.0.0.10
If a subfield is used without the parent field, the query will be expanded to include all valid parent fields. For instance, the subfield ip could expand to dst.ip, src.ip, and a number of others. The block below shows the complete expansion for the ip field in a dns event.
// original query
dns:ip = 10.0.0.10
// rewritten to expand the unspecified parent field
dns:src.ip = 10.0.0.10 OR dns:dst.ip = 10.0.0.10 OR dns:answers.ip = 10.0.0.10
Event-type and field expansion can be applied to the same query. For example, if we simply specified the ip field, the query engine would expand to all possible parent fields in all possible event types.
// original query
ip = 10.0.0.10
// complete expansion of event type and parent field (truncated)
dns:src.ip = '10.0.0.10' OR dns:dst.ip = '10.0.0.10' OR dns:asnwers.ip = 10.0.0.10 OR flow:src.ip = '10.0.0.10' OR flow:dst.ip = '10.0.0.10'
Synthetic fields
A synthetic field is a field that doesn't exist in an event record, i.e. it isn't static. Synthetic fields are dynamically evalutated and converted into static values before your IQL query is run against the event data store. This enables more robust capabilities that aren't possible with a simple query of static values.
Synthetic fields begin with a $. The example query below demonstrates the $device synthetic field, which enables a user to search for a source or destination device by hostname or MAC address instead of just the observed IP address. The hostname is evaluated behind the scenes to produce a large array of IP addresses and valid time ranges, which are then used to query the event data store.
src.$device.hostname = 'FinanceWks008' and dst.internal = false