This example will show how the content of the requests and responses for the web API can be logged. Because the content of API requests and responses may contain sensitive data, this information is can only be logged to a table in the ODS database to which that handled the API call was directed. Logging the full content of the API transactions Furthermore, the duration for which this logging can be enabled is limited because it has the potential to generate a large amount of data, so the duration for which this can be enabled is limited.
The steps to enable web API request/response logging can be summarized as:
Step 1. Create a table for storing the log in the ODS database
Because the content of an API transaction may contain sensitive data, the only supported storage for logging the content of the transaction is the ODS database which was used to process it. This ensures that the same level of security provided for the data at rest in the ODS database app also applies to the data stored in the log entries as well.The required table can be created by running the following SQL script on the ODS database:
Using MS SQL Server
| Code Block |
|---|
|
CREATE TABLE dbo.RequestResponseContentLog (
Id int IDENTITY (1, 1) NOT NULL,
Date datetime NOT NULL,
Thread varchar(255) NOT NULL,
ApiClientId varchar(255) NULL,
Level varchar(50) NULL,
RequestUrl varchar(255) NULL,
RequestMethod varchar(10) NULL,
ProfilesHeader varchar(255) NULL,
RequestBody nvarchar(max) NULL,
ResponseBody nvarchar(max) NULL,
ResponseMessage varchar(255) NULL,
Exception varchar(2000) NULL
) |
Using PostgreSQL
| Code Block |
|---|
|
CREATE TABLE edfi.RequestResponseContentLog (
Id SERIAL PRIMARY KEY NOT NULL,
Date TIMESTAMP NOT NULL,
Thread varchar (255) NOT NULL,
ApiClientId varchar (255) NULL,
Level varchar (50) NULL,
RequestUrl varchar (255) NULL,
RequestMethod varchar (10) NULL,
ProfilesHeader varchar (255) NULL,
RequestBody varchar NULL,
ResponseBody varchar NULL,
ResponseMessage varchar (255) NULL,
Exception varchar (2000) NULL
)
|
Step 2. Update the EdFi.Ods.WebApi log4net configuration
The log4net configuration for EdFi.Ods.WebApi needs to must be updated to include the settings needed for request/response content logging.
Add the following elements inside the root <log4net> element of the active log4net configuration file in use for the web API, typically either log4net.development.config or log4net.config.
| Info |
|---|
By default, only requests that result in an exception are logged. To log the request/response content of all calls to the web API, change <threshold value="ERROR" /> to <threshold value="INFO" /> in both of the places, it appears in the the log4net configuration. |
Using MS SQL Server
| Code Block |
|---|
|
<logger name="RequestResponseContentLogger" additivity="false">
<level value="ERROR" />
<appender-ref ref="RequestResponseContentAppender" />
</logger>
<appender name="RequestResponseContentAppender" type="log4net.Appender.AdoNetAppender">
<bufferSize value="0" />
<threshold value="ERROR" />
<connectionType value="System.Data.SqlClient.SqlConnection, System.Data, Version=1.0.3300.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
<commandText value="INSERT INTO RequestResponseContentLog ([Date],[Thread],[ApiClientId],[Level],[RequestUrl],[RequestMethod],[ProfilesHeader],[RequestBody],[ResponseBody],[ResponseMessage],[Exception]) VALUES (@log_date, @thread, @api_client_id, @log_level, @request_url, @request_method, @profiles_header, @request_body, @response_body, @response_message, @exception)" />
<parameter>
<parameterName value="@log_date" />
<dbType value="DateTime" />
<layout type="log4net.Layout.RawTimeStampLayout" />
</parameter>
<parameter>
<parameterName value="@thread" />
<dbType value="String" />
<size value="255" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%thread" />
</layout>
</parameter>
<parameter>
<parameterName value="@api_client_id" />
<dbType value="String" />
<size value="255" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%property{ApiClientId}" />
</layout>
</parameter>
<parameter>
<parameterName value="@log_level" />
<dbType value="String" />
<size value="50" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%level" />
</layout>
</parameter>
<parameter>
<parameterName value="@request_url" />
<dbType value="String" />
<size value="255" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%property{RequestUrlWithQueryString}" />
</layout>
</parameter>
<parameter>
<parameterName value="@request_method" />
<dbType value="String" />
<size value="10" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%property{RequestMethod}" />
</layout>
</parameter>
<parameter>
<parameterName value="@profiles_header" />
<dbType value="String" />
<size value="255" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%property{ProfilesHeader}" />
</layout>
</parameter>
<parameter>
<parameterName value="@request_body" />
<dbType value="String" />
<size value="-1" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%property{RequestBody}" />
</layout>
</parameter>
<parameter>
<parameterName value="@response_body" />
<dbType value="String" />
<size value="-1" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%property{ResponseBody}" />
</layout>
</parameter>
<parameter>
<parameterName value="@response_message" />
<dbType value="String" />
<size value="255" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%message" />
</layout>
</parameter>
<parameter>
<parameterName value="@exception" />
<dbType value="String" />
<size value="2000" />
<layout type="log4net.Layout.ExceptionLayout" />
</parameter>
</appender> |
Using PostgreSQL
| Code Block |
|---|
|
<logger name="RequestResponseContentLogger" additivity="false">
<level value="ERROR" />
<appender-ref ref="RequestResponseContentAppender" />
</logger>
<appender name="RequestResponseContentAppender" type="log4net.Appender.AdoNetAppender">
<bufferSize value="0" />
<threshold value="ERROR" />
<connectionType value="Npgsql.NpgsqlConnection, Npgsql" />
<commandText value="INSERT INTO edfi.RequestResponseContentLog (date,thread,apiclientid,level,requesturl,requestmethod,profilesheader,requestbody,responsebody,responsemessage,exception) VALUES (@log_date, @thread, @api_client_id, @log_level, @request_url, @request_method, @profiles_header, @request_body, @response_body, @response_message, @exception)" />
<parameter>
<parameterName value="@log_date" />
<dbType value="DateTime" />
<layout type="log4net.Layout.RawTimeStampLayout" />
</parameter>
<parameter>
<parameterName value="@thread" />
<dbType value="String" />
<size value="255" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%thread" />
</layout>
</parameter>
<parameter>
<parameterName value="@api_client_id" />
<dbType value="String" />
<size value="255" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%property{ApiClientId}" />
</layout>
</parameter>
<parameter>
<parameterName value="@log_level" />
<dbType value="String" />
<size value="50" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%level" />
</layout>
</parameter>
<parameter>
<parameterName value="@request_url" />
<dbType value="String" />
<size value="255" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%property{RequestUrlWithQueryString}" />
</layout>
</parameter>
<parameter>
<parameterName value="@request_method" />
<dbType value="String" />
<size value="10" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%property{RequestMethod}" />
</layout>
</parameter>
<parameter>
<parameterName value="@profiles_header" />
<dbType value="String" />
<size value="255" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%property{ProfilesHeader}" />
</layout>
</parameter>
<parameter>
<parameterName value="@request_body" />
<dbType value="String" />
<size value="-1" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%property{RequestBody}" />
</layout>
</parameter>
<parameter>
<parameterName value="@response_body" />
<dbType value="String" />
<size value="-1" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%property{ResponseBody}" />
</layout>
</parameter>
<parameter>
<parameterName value="@response_message" />
<dbType value="String" />
<size value="255" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%message" />
</layout>
</parameter>
<parameter>
<parameterName value="@exception" />
<dbType value="String" />
<size value="2000" />
<layout type="log4net.Layout.ExceptionLayout" />
</parameter>
</appender> |
Step 3. Set the logging duration
Update the following setting in the appsettings.json for EdFi.Ods.WebApi to reflect the number of minutes request/response content should be logged. The measurement of this logging duration begins when the first web API request is received after the application startup of the application.
| Code Block |
|---|
|
"LogRequestResponseContentForMinutes": 0 |
...