Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Next »

This example will show how the content of the requests and responses for the web API can be logged. Because the content of API requests and responses may contain sensitive data, this information can only be logged to a table in the ODS database that handled the API call. Logging the full content of the API transactions has the potential to generate a large amount of data, so the duration for which this can be enabled is limited.

The steps can be summarized as:

Step 1. Create a table for storing the log in the ODS database

Because the content of an API transaction may contain sensitive data, the only supported storage for logging the content of the transaction is the ODS database which was used to process it. This ensures that the same level of security provided for the data at rest in the ODS database applies to the data stored in the log entries as well.

The required table can be created by running the following SQL script on the ODS database:

Using MS SQL Server

CREATE TABLE dbo.RequestResponseContentLog (
    Id int IDENTITY (1, 1) NOT NULL,
    Date datetime NOT NULL,
	Thread varchar(255) NOT NULL,
    ApiClientId varchar(255) NULL,
    Level varchar(50) NULL,
	RequestUrl varchar(255) NULL,
	RequestMethod varchar(10) NULL,
	ProfilesHeader varchar(255) NULL,
	RequestBody nvarchar(max) NULL,
	ResponseBody nvarchar(max) NULL,
    ResponseMessage varchar(255) NULL,
    Exception varchar(2000) NULL
)

Using PostgreSQL

CREATE TABLE edfi.RequestResponseContentLog (
Id SERIAL PRIMARY KEY NOT NULL,
Date TIMESTAMP NOT NULL,
Thread varchar (255) NOT NULL,
ApiClientId varchar (255) NULL,
Level varchar (50) NULL,
RequestUrl varchar (255) NULL,
RequestMethod varchar (10) NULL,
ProfilesHeader varchar (255) NULL,
RequestBody varchar NULL,
ResponseBody varchar NULL,
ResponseMessage varchar (255) NULL,
Exception varchar (2000) NULL
)

Step 2. Update the EdFi.Ods.WebApi log4net configuration

The log4net configuration for EdFi.Ods.WebApi needs to be updated to include the settings needed for request/response content logging.

Add the following elements inside the root <log4net> element of the log4net configuration file in use for the web API, typically either log4net.development.config or log4net.config.

By default, only requests that result in an exception are logged. To log the request/response content of all calls to the web API, change <threshold value="ERROR" /> to <threshold value="INFO" /> in both of the places it appears in the the log4net configuration.

Using MS SQL Server

  <logger name="RequestResponseContentLogger" additivity="false">
	<level value="ERROR" />
	<appender-ref ref="RequestResponseContentAppender" />
  </logger>
  <appender name="RequestResponseContentAppender" type="log4net.Appender.AdoNetAppender">
    <bufferSize value="0" />
    <threshold value="ERROR" />
    <connectionType value="System.Data.SqlClient.SqlConnection, System.Data, Version=1.0.3300.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
    <commandText value="INSERT INTO RequestResponseContentLog ([Date],[Thread],[ApiClientId],[Level],[RequestUrl],[RequestMethod],[ProfilesHeader],[RequestBody],[ResponseBody],[ResponseMessage],[Exception]) VALUES (@log_date, @thread, @api_client_id, @log_level, @request_url, @request_method, @profiles_header, @request_body, @response_body, @response_message, @exception)" />
    <parameter>
        <parameterName value="@log_date" />
        <dbType value="DateTime" />
        <layout type="log4net.Layout.RawTimeStampLayout" />
    </parameter>
    <parameter>
        <parameterName value="@thread" />
        <dbType value="String" />
        <size value="255" />
        <layout type="log4net.Layout.PatternLayout">
            <conversionPattern value="%thread" />
        </layout>
    </parameter>
    <parameter>
	    <parameterName value="@api_client_id" />
	    <dbType value="String" />
	    <size value="255" />
	    <layout type="log4net.Layout.PatternLayout">
		    <conversionPattern value="%property{ApiClientId}" />
	    </layout>
    </parameter>
    <parameter>
        <parameterName value="@log_level" />
        <dbType value="String" />
        <size value="50" />
        <layout type="log4net.Layout.PatternLayout">
            <conversionPattern value="%level" />
        </layout>
    </parameter>
    <parameter>
	    <parameterName value="@request_url" />
	    <dbType value="String" />
	    <size value="255" />
	    <layout type="log4net.Layout.PatternLayout">
		    <conversionPattern value="%property{RequestUrlWithQueryString}" />
	    </layout>
    </parameter>
    <parameter>
	    <parameterName value="@request_method" />
	    <dbType value="String" />
	    <size value="10" />
	    <layout type="log4net.Layout.PatternLayout">
		    <conversionPattern value="%property{RequestMethod}" />
	    </layout>
    </parameter>
    <parameter>
	    <parameterName value="@profiles_header" />
	    <dbType value="String" />
	    <size value="255" />
	    <layout type="log4net.Layout.PatternLayout">
		    <conversionPattern value="%property{ProfilesHeader}" />
	    </layout>
    </parameter>
    <parameter>
        <parameterName value="@request_body" />
        <dbType value="String" />
        <size value="-1" />
        <layout type="log4net.Layout.PatternLayout">
            <conversionPattern value="%property{RequestBody}" />
        </layout>
    </parameter>
    <parameter>
	    <parameterName value="@response_body" />
	    <dbType value="String" />
	    <size value="-1" />
	    <layout type="log4net.Layout.PatternLayout">
		    <conversionPattern value="%property{ResponseBody}" />
	    </layout>
    </parameter>
    <parameter>
        <parameterName value="@response_message" />
        <dbType value="String" />
        <size value="255" />
        <layout type="log4net.Layout.PatternLayout">
            <conversionPattern value="%message" />
        </layout>
    </parameter>
    <parameter>
        <parameterName value="@exception" />
        <dbType value="String" />
        <size value="2000" />
        <layout type="log4net.Layout.ExceptionLayout" />
    </parameter>
  </appender>

Using PostgreSQL

  <logger name="RequestResponseContentLogger" additivity="false">
	<level value="ERROR" />
	<appender-ref ref="RequestResponseContentAppender" />
  </logger>
  <appender name="RequestResponseContentAppender" type="log4net.Appender.AdoNetAppender">
    <bufferSize value="0" />
    <threshold value="ERROR" />
     <connectionType value="Npgsql.NpgsqlConnection, Npgsql" />
    <commandText value="INSERT INTO edfi.RequestResponseContentLog (date,thread,apiclientid,level,requesturl,requestmethod,profilesheader,requestbody,responsebody,responsemessage,exception) VALUES (@log_date, @thread, @api_client_id, @log_level, @request_url, @request_method, @profiles_header, @request_body, @response_body, @response_message, @exception)" />
    <parameter>
        <parameterName value="@log_date" />
        <dbType value="DateTime" />
        <layout type="log4net.Layout.RawTimeStampLayout" />
    </parameter>
    <parameter>
        <parameterName value="@thread" />
        <dbType value="String" />
        <size value="255" />
        <layout type="log4net.Layout.PatternLayout">
            <conversionPattern value="%thread" />
        </layout>
    </parameter>
    <parameter>
	    <parameterName value="@api_client_id" />
	    <dbType value="String" />
	    <size value="255" />
	    <layout type="log4net.Layout.PatternLayout">
		    <conversionPattern value="%property{ApiClientId}" />
	    </layout>
    </parameter>
    <parameter>
        <parameterName value="@log_level" />
        <dbType value="String" />
        <size value="50" />
        <layout type="log4net.Layout.PatternLayout">
            <conversionPattern value="%level" />
        </layout>
    </parameter>
    <parameter>
	    <parameterName value="@request_url" />
	    <dbType value="String" />
	    <size value="255" />
	    <layout type="log4net.Layout.PatternLayout">
		    <conversionPattern value="%property{RequestUrlWithQueryString}" />
	    </layout>
    </parameter>
    <parameter>
	    <parameterName value="@request_method" />
	    <dbType value="String" />
	    <size value="10" />
	    <layout type="log4net.Layout.PatternLayout">
		    <conversionPattern value="%property{RequestMethod}" />
	    </layout>
    </parameter>
    <parameter>
	    <parameterName value="@profiles_header" />
	    <dbType value="String" />
	    <size value="255" />
	    <layout type="log4net.Layout.PatternLayout">
		    <conversionPattern value="%property{ProfilesHeader}" />
	    </layout>
    </parameter>
    <parameter>
        <parameterName value="@request_body" />
        <dbType value="String" />
        <size value="-1" />
        <layout type="log4net.Layout.PatternLayout">
            <conversionPattern value="%property{RequestBody}" />
        </layout>
    </parameter>
    <parameter>
	    <parameterName value="@response_body" />
	    <dbType value="String" />
	    <size value="-1" />
	    <layout type="log4net.Layout.PatternLayout">
		    <conversionPattern value="%property{ResponseBody}" />
	    </layout>
    </parameter>
    <parameter>
        <parameterName value="@response_message" />
        <dbType value="String" />
        <size value="255" />
        <layout type="log4net.Layout.PatternLayout">
            <conversionPattern value="%message" />
        </layout>
    </parameter>
    <parameter>
        <parameterName value="@exception" />
        <dbType value="String" />
        <size value="2000" />
        <layout type="log4net.Layout.ExceptionLayout" />
    </parameter>
  </appender>

Step 3. Set the logging duration

Update the following setting in the appsettings.json for EdFi.Ods.WebApi to reflect the number of minutes request/response content should be logged. The measurement of this logging duration begins when the first web API request is received after the startup of the application. 

"LogRequestResponseContentForMinutes": 0




  • No labels