Enabling File Integrity Monitoring on Windows with Osquery and EclecticIQ Endpoint Response

File Integrity Monitoring (FIM) is a security control that helps organizations ensure the integrity of their files and systems by monitoring changes to files and directories. FIM is an important security control needed for almost all kinds of compliance requirements, like PCI DSS, HIPAA, GDPR and ISO. The aim of FIM is to verify the integrity of application software files to determine if they have been tampered with or if a fraud has occurred by comparing them with a baseline. FIM solutions are also used to monitor activities on sensitive files (e.g. configuration as well as content files) and can trigger alerts based on rules around the access. FIM solutions use different methods, such as comparing file attributes (e.g., file size, timestamps, hashes) to detect changes, monitoring file access and modification events, or using machine learning to detect anomalous behaviour. Thereby, having a FIM solution is not only important from the standpoint of a compliance requirement, it also is an essential toolkit for security monitoring.

Using FIM with osquery

Osquery achieves FIM through the evented tables. The evented tables capture the events on a pre-determined set of directories (or files) and thus osquery agent captures the changes to the files being monitored. For a long time , this capability was available only on Linux and macOS until a much later version when FIM was also available using osquery's ntfs_journal_events table created on Windows.

This table provides monitoring capabilities for file system events on Windows and is a good start for getting basic FIM capabilities on Windows. There are limitations to what osquery can achieve. Looking at the schema of the ntfs_journal_events, it gives a detailed information on the target file and the type of modification event. Unfortunately, it doesn't provide any info on which process did the action. It also lacks a unique file identifier like file hash (e.g. md5 or sha256). The reason for that being 'under-the-hoods' implementation of the table as it relies on monitoring NTFS journal. This provides an advantage of not having to install a kernel component (like a mini-filter driver) but that comes at a cost of missing important context around the event. And contextualization of an event can be very critical, more often than not (even chatGPT thinks so).

Figure 1: ChatGPT on FIM.

Using FIM in osquery with EclecticIQ Extension

Using EclecticIQ Extension to osquery creates tables based on a kernel module that extends osquery on Windows with a wide range of real-time events as they happen on the device. Each event is also enriched with the contextual information around the event that includes the process that triggered the event, the user context, the file hashes (for file events) and the date-time stamps. Each event type is reported in an extended table and because it natively fits with osquery, all the SQL syntax and structures get honoured. The file events are recorded in table known as win_file_events and the schema for reporting file events is as below.

Figure 2: win_file_events table schema.

Activities collected in win_file_events, unlike ntfs_journal_events, are not restricted to drives with NTFS file systems only. Thus, collecting data using win_file_events also brings the file activity done on other types of storage like the attached USB drives and the network mounted drives. As the events are recorded in real-time, this can form a much more sophisticated foundation for configuring FIM on Windows with osquery. With each file event, what also gets collected (and reported) is the file hash and the process that triggered the file activity. The EclecticIQ Extension tracks each process by its GUID.

Configuring FIM in EclecticIQ Endpoint Response

EclecticIQ ER application packages osquery and the extension together with a web-based application to enable the management of the agents. FIM on EclecticIQ Endpoint Response agent (osquery and the extension) can be configured just like file events are configured on other operating systems (Linux/MacOS) that are natively available in osquery. By default on Windows, the EclecticIQ Endpoint Response application enables file events on all the directories and instead relies on the EclecticIQ agent's filters to do a more targeted collection of events. These filters can also be applied to file events based on their contextual criteria as explained in our previous article.

Once the agent is configured, it would start capturing file events and they are pushed to the EclecticIQ Endpoint Response server that has pre-built queries for the evented tables. The “Rule” interface of the EclecticIQ Endpoint Response server can be utilized to generate alerts on file events of interest. The following figure shows how to configure a rule to generate an alert on every file_write event*.

*This rule is only for demonstration purposes and can be very generic. Additional conditions should be added to get high-quality alerts.

Figure 3: Configuring a FIM rule in EclecticIQ Endpoint Response.

Once configured, the rule can be viewed in the “ER Rules” list.

Figure 4: Rule listing EclecticIQ Endpoint Response.

Once a rule is activated, every time the agent captures the matching behaviour, it will create an alert in the EclecticIQ Endpoint Response application. The EclecticIQ Endpoint Response application provides multiple options to triage the alerts, from monitoring raw alert to getting its full context and timeline of activities around it. The following figures provide the overview of EclecticIQ Endpoint Response capabilities to triage an alert.

Figure 5: Alert created by the FIM rule and timeline of activity on the endpoint.

Figure 6: Raw event on the file that matched the criteria.

Figure 7: Process tree graph of the process triggering the alert.

Given the extensive context around the FIM event that EclecticIQ Endpoint Response provides, it can pretty much serve any requirement of a FIM use case. But wait, it gets even better. Given that the entire EclecticIQ Endpoint Response agent leverages osquery’s SQL form factor for data collection, interesting set of live queries can be sent to the agent to connect, correlate and filter the data as to the needs of an analysts or an administrator tasked with FIM.

Figure 8: Example of leveraging osquery SQL to get event corelation.

The EclecticIQ Endpoint Response’s extension that collects the real-time file events on Windows and with native integration via osquery into file events on MacOS and Linux, makes EclecticIQ ER as a much more powerful application for getting cross-platform FIM with osquery.  

You might also be interested in:

If Infosec Was a Supermarket Business

Dark Pink APT Group Strikes Government Entities in South Asian Countries

Comparing Sysmon and EclecticIQ Endpoint Response - Event Filters

About Endpoint Security Solution Assessment

The assessment should cover all aspects of our traditional People, Process, and Technology Framework. Check out the whitepaper on “5 Questions to Ask About Your EDR” to help you make an informed decision.

About EclecticIQ Endpoint Response

EclecticIQ Endpoint Response solution offers unapparelled visibility into endpoint telemetry - by using the proven open-source telemetry tool osquery as a foundation and adding our own custom extensions on top, achieving in a single agent what would otherwise require multiple tools running in unison. Interested to learn more, feel free to contact us.

About EclecticIQ Endpoint Response Community Edition

The EclecticIQ Community Edition platform is a sophisticated and flexible endpoint monitoring and response platform, based on the osquery agent. It provides endpoint monitoring and visibility, threat detection, and incident response for Security Operating Centers (SOCs). Download it on Github.