Log Analysis For Cyber Attack Detection using AI

Abstract

The rapid escalation of cyber threats against networked computing environments demands automated, intelligent mechanisms for real-time log monitoring and multi-format threat classification. Existing lightweight Security Information and Event Management (SIEM) platforms are constrained by their reliance on manually curated detection signatures and by documented recall failures in NSL-KDD-trained machine learning classifiers. This paper presents UpgradedAMIDES, a hybrid XGBoost ensemble classifier integrated with a TF-IDF character n-gram feature extractor, constituting the core detection engine of a self-hosted, full-stack SIEM platform. A principal contribution is the systematic identification and correction of a BRUTE_FORCE recall failure present in prior AMIDES-based implementations — arising from erroneous reliance on the num_failed_logins feature, which carries negligible discriminative signal for Remote-to-Local (R2L) attack records in NSL-KDD. Five data-driven sub-type score features encoding actual R2L behavioral signatures are introduced to address this failure. The platform further supports unified normalization of Windows Event Logs, network flow records, and syslog streams into a canonical feature representation, an autonomous remediation engine, a 14-endpoint REST API, and a nine-page React/TypeScript dashboard. Experimental results demonstrate that the engineered feature set substantially improves recall for the BRUTE_FORCE threat class and that the entropy suppression preprocessing eliminates false-positive inflation on routine audit events.

Keywords