AWS Introduces Amazon Security Lake and AWS Clean Rooms

(VideoFlow/Shutterstock)

AWS has introduced two new security services at re:Invent 2022: Amazon Security Lake and AWS Clean Rooms.

Amazon Security Lake is a service that automatically centralizes an organization’s security data from cloud and on-premises sources into a purpose-built data lake in a customer’s AWS account, says AWS. The data is then managed throughout its lifecycle with customizable data retention settings while converting incoming security data into the Apache Parquet format, conforming it to the Open Cybersecurity Schema Framework (OCSF) open standard. AWS notes this will make data governance easier by automatically normalizing security data from AWS and combining it with dozens of pre-integrated, third-party enterprise security data sources.

AWS says it saw a need for this service through its customers’ desire for greater visibility into security activity across their entire organizations to proactively identify potential threats and vulnerabilities, assess security alerts, respond accordingly, and help prevent future security events. Many companies must rely on using log and event data for this task, often from many disparate sources that frequently have incompatible data formats. Collecting and normalizing data into a consistent format to be analyzed for threats is the goal, and while some companies have invested in a central repository for their security data in a data lake, these systems can be costly and time-consuming to build.

“Security analysts and engineers can use Amazon Security Lake to aggregate, manage, and optimize large volumes of disparate log and event data to enable faster threat detection, investigation, and incident response to effectively address potential issues quickly, while continuing to utilize their preferred analytics tools,” the company stated in a release.

This graphic shows the automated end-to-end data governance process enabled by Amazon Security Lake. Click to enlarge. Source: AWS

Amazon Security Lake can be created in just a few clicks, claims AWS. After the service is set up and connected to selected data sources, Amazon Security Lake automatically builds a security data lake in a customer-selected region where it begins to aggregate, manage, and optimize the incoming data in an automated end-to-end process. The security data lake is built using Amazon Simple Storage Service (Amazon S3) and AWS Lake Formation. Customers can use preferred security and analytics once their data is ingested and normalized, including Amazon Athena, Amazon OpenSearch, and Amazon SageMaker, along with third-party solutions such as IBM, Splunk, or Sumo Logic, the company says.

“Customers must be able to quickly detect and respond to security risks so they can take swift action to secure data and networks, but the data they need for analysis is often spread across multiple sources and stored in a variety of formats. Customers tell us they want to take action on this data faster to improve their security posture, but the process of collecting, normalizing, storing, and managing this data is complex and time consuming,” said Jon Ramsey, vice president for Security Services at AWS. “Amazon Security Lake lets customers of all sizes securely set up a security data lake with just a few clicks to aggregate logs and event data from dozens of sources, normalize it to conform with the OCSF standard, and make it more broadly usable so customers can take action quickly using their security tools of choice. With Amazon Security Lake, customers get superior visibility and control, with help from the largest ecosystem of security partners and solutions.”

Amazon Security Lake is now available in preview in US East (N. Virginia), US East (Ohio), US West (Oregon), Asia Pacific (Sydney), Asia Pacific (Tokyo), Europe (Frankfurt), and Europe (Ireland), with availability in additional AWS Regions coming soon.

AWS Clean Rooms was announced at re:Invent 2022. Source: AWS

The second new service is called AWS Clean Rooms, a new analytics service that enables secure analysis and collaboration on combined datasets without revealing underlying data. AWS claims customers across multiple industries can create a secure data clean room in minutes and collaborate with any other company in the AWS Cloud to generate unique insights about advertising campaigns, investment decisions, clinical research, and more.

Built-in data access controls are provided to protect sensitive data, including query controls, query output restrictions, query logging, and cryptographic computing tools that keep data encrypted, even as queries are processed, notes AWS.

AWS was again responding to the needs of its customers who want to complement or combine their data with that of their external business partners for a more comprehensive view of their business, such as in the advertising industry, where multiple brands and media publishers collaborate using data stored in various channels and applications. Protecting sensitive information while reducing the sharing of raw data is the goal, and sharing user-level data along with contractual agreements meant to prevent misuse of this data has often been a solution.

Instead of relying on the honor system, data clean rooms allow multiple parties to combine and analyze data in a protected environment, according to AWS. But just like security data lakes, these are challenging to build as they include specialized tools, complex privacy controls, and months of development time for customization. Data movement is also a concern, and adding new users sometimes means moving data out of the environment onto other platforms.

(Michael-Vi/Shutterstock)

AWS Clean Rooms offers creation of data clean rooms in minutes, AWS claims. Customers can choose partners they want to collaborate with from the AWS Management Console while selecting datasets and configuring restrictions for participants. “With AWS Clean Rooms, customers can easily collaborate with companies that are already using AWS, without moving data out of their AWS environment or loading it into another platform. When customers run queries, AWS Clean Rooms reads data where it lives and applies built-in analysis rules to help them maintain control over their data,” AWS said in a release.

“Customers tell us they want to collaborate more safely and securely with their partners in areas like advertising, media, financial services, and life sciences. However, the data they need to do this is fragmented across data stores and applications belonging to different partners,” said Dilip Kumar, vice president of AWS Applications. “AWS Clean Rooms helps customers and their partners to better analyze and collaborate on their data on AWS. With the launch of AWS Clean Rooms, we are making it easier, simpler, and more secure for multiple companies to share and analyze combined datasets to generate new insights that they could not do on their own. Using AWS Clean Rooms, customers can collaborate on a range of tasks, such as more effectively generating advertising campaign insights and analyzing investment data while improving data security.”

AWS Clean Rooms will be available as a self-contained offering or as part of AWS for Advertising and Marketing in early 2023 in US East (Ohio), US East (N. Virginia), US West (Oregon), Asia Pacific (Seoul), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), Europe (Frankfurt), Europe (Ireland), Europe (London), and Europe (Stockholm).

Related Items:

AWS Bolsters Glue ETL Tool with Data Observability, Ray Support

AWS Seeks an End to ETL

AWS Introduces a Flurry of New EC2 Instances at re:Invent