I have encountered the following error on numerous occasions when installing ArcSight Connectors:
Error: missing 'server' JVM at '$Connector_Install_Location$/current/jre/lib/i386/server/libjvm.so'
The fix is really easy. Install the Open JDK for 1.7.0
yum install Java-1.7.0-openjdk
Once installed delete the jre folder in the Connector you are deploying. The jre folder is located at:
$Connector_Install_Location$/current/jre/
so do:
rm -rf $Connector_Install_Location$/current/jre
Once deleted, copy the Open JDK files over to the Connector folder with:
cp -r /usr/lib/jvm/jre-1.7.0-openjdk.x86_64/ $Connector_Install_Location$/current/jre
You can then run the agent setup without issues.
Wednesday 25 October 2017
ArcSight Connector: Standard DNS Connector Deployment
An out of the box Windows DNS Connector deployment does not capture the deviceHostName of the server which you are reading from. We can fix this with a couple of tweaks:
1) Ensure the DNS server IP address exists in the folder structure. When configuring and sharing the DNS logs, set up the file share as follows:
\WindowsServer\deviceIP\anotherfolder\
The folder structure doesn't matter too much as long as the deviceIP exists. An example would be
\\windowshostname\1.1.1.1\logs\
2) in the agent.properties configuration, ensure the following is set:
agents[1].foldertable[0].extractfieldnames=deviceHostName
agents[1].foldertable[0].extractregex=/opt/mnt/(.*)/dns.log
agents[1].foldertable[0].extractsource=File Path
agents[1].foldertable[0].fixedlinelength=-1
agents[1].foldertable[0].fixedlinelengthcontains=Fixed Number Of Characters
agents[1].foldertable[0].folder=\WindowsServer\(.*)\anotherfolder\dns.log
agents[1].foldertable[0].followexternalrotation=true
This tells the Connector to extract the deviceHostName from the File Path. The RegEx string will extract the deviceIP (in this case 1.1.1.1) and add it to the event data.
1) Ensure the DNS server IP address exists in the folder structure. When configuring and sharing the DNS logs, set up the file share as follows:
\WindowsServer\deviceIP\anotherfolder\
The folder structure doesn't matter too much as long as the deviceIP exists. An example would be
\\windowshostname\1.1.1.1\logs\
2) in the agent.properties configuration, ensure the following is set:
agents[1].foldertable[0].extractfieldnames=deviceHostName
agents[1].foldertable[0].extractregex=/opt/mnt/(.*)/dns.log
agents[1].foldertable[0].extractsource=File Path
agents[1].foldertable[0].fixedlinelength=-1
agents[1].foldertable[0].fixedlinelengthcontains=Fixed Number Of Characters
agents[1].foldertable[0].folder=\WindowsServer\(.*)\anotherfolder\dns.log
agents[1].foldertable[0].followexternalrotation=true
This tells the Connector to extract the deviceHostName from the File Path. The RegEx string will extract the deviceIP (in this case 1.1.1.1) and add it to the event data.
ArcSight Connector: Standard DHCP Connector Deployment
An out of the box Windows DHCP Connector deployment does not capture the deviceAddress of the server which you are reading from. We can fix this with a couple of tweaks:
1) Ensure the DHCP server IP address exists in the folder structure. When configuring and sharing the DHCP logs, set up the file share as follows:
\WindowsServer\deviceIP\anotherfolder\
The folder structure doesn't matter too much as long as the deviceIP exists. An example would be
\\windowshostname\1.1.1.1\logs\
2) in the agent.properties configuration, ensure the following is set:
agents[0].extractfieldnames=deviceAddress
agents[0].extractregex=\\S+/(\\d+\\.\\d+\\.\\d+\\.\\d+)\\S+
agents[0].extractsource=File Path
agents[0].usefieldextractor=true
agents[0].usenonlockingwindowsfilereader=true
This tells the Connector to extract the deviceAddress from the File Path. The RegEx string will extract the deviceIP (in this case 1.1.1.1) and add it to the event data.
1) Ensure the DHCP server IP address exists in the folder structure. When configuring and sharing the DHCP logs, set up the file share as follows:
\WindowsServer\deviceIP\anotherfolder\
The folder structure doesn't matter too much as long as the deviceIP exists. An example would be
\\windowshostname\1.1.1.1\logs\
2) in the agent.properties configuration, ensure the following is set:
agents[0].extractfieldnames=deviceAddress
agents[0].extractregex=\\S+/(\\d+\\.\\d+\\.\\d+\\.\\d+)\\S+
agents[0].extractsource=File Path
agents[0].usefieldextractor=true
agents[0].usenonlockingwindowsfilereader=true
This tells the Connector to extract the deviceAddress from the File Path. The RegEx string will extract the deviceIP (in this case 1.1.1.1) and add it to the event data.
ArcSight ESM: Re-Enable a disabled user in ESM
ESM user accounts can be locked for a variety of reasons. A quick way to unlock them is to SSH to the ESM server and run the following:
$ArcSight_ESM_Location$\manager\bin\arcsight reenableuser username
The username being the users ESM username.
If the admin user account is locked then I would run:
$ArcSight_ESM_Location$\manager\bin\arcsight reenableuser admin
$ArcSight_ESM_Location$\manager\bin\arcsight reenableuser username
The username being the users ESM username.
If the admin user account is locked then I would run:
$ArcSight_ESM_Location$\manager\bin\arcsight reenableuser admin
ArcSight Connector: Moving Data via a Map file
Map files can be used to run additional processing on event data. Sometimes I want to keep the original field untouched but run run some extra processing or extract information into another field.
The location of this file will be $ArcSight_Connector_Location$\current\user\agent\map\
The file will be titled map.X.properties, X being the next iteration of map file you have. By default 1 and 2 already exist so it's safe to name the file map.3.properties
In this scenario I want to move data from deviceCustomString1 to requestContext. I have some firewall events which present the Context in deviceCustomString4 but my Rules are looking for this data in requestContext. The file looks like this:
set.expr(deviceCustomString1).event.requestContext
"__regexToken(deviceCustomString1,""(.*)"")"
The .* in red is the RegEx string. This example will take everything in deviceCustomString1 and dump it into requestContext.
The location of this file will be $ArcSight_Connector_Location$\current\user\agent\map\
The file will be titled map.X.properties, X being the next iteration of map file you have. By default 1 and 2 already exist so it's safe to name the file map.3.properties
In this scenario I want to move data from deviceCustomString1 to requestContext. I have some firewall events which present the Context in deviceCustomString4 but my Rules are looking for this data in requestContext. The file looks like this:
set.expr(deviceCustomString1).event.requestContext
"__regexToken(deviceCustomString1,""(.*)"")"
The .* in red is the RegEx string. This example will take everything in deviceCustomString1 and dump it into requestContext.
How to split a zip file into chunks on Linux
Sometimes I need to upload log files to a vendor. For some reason large vendors like to add file size limits to their uploads. In a large environment this makes it hard to upload logs.
A way around this is to zip a folder and split it into smaller chunks of a pre defined size.
Lets say a vendor imposes a maximum file upload size of 85MB. we could use the zip command and tell it to create a new file every 85MB with the following:
zip -r -s 85m new-zip-name.zip /tmp/logfiles/*
new-zip-name.zip can be anything you want as long as it ends in .zip. The resulting files will have a number appended to them.
/tmp/logfiles/* is the directory you want to zip up.
A way around this is to zip a folder and split it into smaller chunks of a pre defined size.
Lets say a vendor imposes a maximum file upload size of 85MB. we could use the zip command and tell it to create a new file every 85MB with the following:
zip -r -s 85m new-zip-name.zip /tmp/logfiles/*
new-zip-name.zip can be anything you want as long as it ends in .zip. The resulting files will have a number appended to them.
/tmp/logfiles/* is the directory you want to zip up.
How to Tar and Gzip a directory
TAR is short for Tape Archive and is often refereed to as a tarball. Think of it as a Windows zip file without the compression.
GZIP is short for GNU Zip. It is used to compress a single file on the Linux platform.
Together, you can TAR a folder into a single file and then reduce the size of the file with Gzip. This makes it easy to transfer around a network or upload it somewhere else.
1) Navigate to the folder which contains the directory.
2) Run the following command:
tar -zcvf new-archive-name.tar.gz directory_name
new-archive-name.tar.gz can be anything providing it ends in tar.gz
directory_name is the name of the directory you are trying to compress. If I wanted to TAR and Gzip the /tmp/ folder, I would run:
tar -zcvf TempFiles.tar.gz /tmp
GZIP is short for GNU Zip. It is used to compress a single file on the Linux platform.
Together, you can TAR a folder into a single file and then reduce the size of the file with Gzip. This makes it easy to transfer around a network or upload it somewhere else.
1) Navigate to the folder which contains the directory.
2) Run the following command:
tar -zcvf new-archive-name.tar.gz directory_name
new-archive-name.tar.gz can be anything providing it ends in tar.gz
directory_name is the name of the directory you are trying to compress. If I wanted to TAR and Gzip the /tmp/ folder, I would run:
tar -zcvf TempFiles.tar.gz /tmp
Subscribe to:
Posts (Atom)