INTRODUCTION

The Sarbanes–Oxley Act of 2002 was signed into law by President Bush on 30th July, 2002, in the wake of corporate scandals of Enron and WorldCom to restore investor confidence in the US capital markets.¹ The law charged the Securities and Exchange Commission (SEC) for implementing its various provisions under a strict timeline and as a result of past audit failures disenfranchised the auditing industry from self-regulation by creating the Public Company Accounting Oversight Board (PCAOB). Since the enactment of the far-reaching governance reforms mandated by the Sarbanes–Oxley Act, Section 404 has consistently dominated the headlines and created an unprecedented amount of backlash as well as counterpoint expressions of support from all those affected by its new internal control certification requirements. Central to the new internal control certifications under Section 404 is the requirement that management and auditors assess the effectiveness of a company's system of internal over financial reporting in accordance with a 'suitable' internal control framework. According to the Section 404 SEC Final Rules² and the PCAOB's Auditing Standard No. 5³ (AS 5), the Internal Control-Integrated Framework (also known as the COSO 1992 to distinguish it from the COSO Committee's other two products namely ERM and Small Business Guidance) developed and issued by the Committee of the Sponsoring Organizations (COSO) of the Treadway Commission meets the stated suitability criteria and can be relied upon both by the management and external auditors for conducting internal control effectiveness evaluations under Section 404 of the Sarbanes–Oxley Act.

Since the passage of the Sarbanes–Oxley Act, a number of surveys and research studies have been conducted to study various aspects of the Section 404 implementation. The majority of these research studies, however, have focused on analysing the extensive costs⁴ flowing from these new compliance requirements. To date, none of the studies has examined how companies and their external auditors are, in fact, utilising the COSO 1992 Framework to assess and report on the effectiveness of a company's internal control over financial reporting (ICFR). This research study fills this void by documenting the current implementation practices at the SEC registrants as they pertain to the use of the COSO 1992 Framework within the context of Section 404(a) control effectiveness reporting requirements. It analyses and reports on the responses of the 374 participants from firms of varying sizes.

DATA COLLECTION AND SAMPLE STATISTICS

This section briefly discusses the data-collection procedures used to obtain data for this research study. It also describes respondent and company demographic information to help the reader better understand the characteristics of the respondent pool.

Consistent with the accepted research protocol, the initial survey was pilot-tested with nine individuals. All of them had considerable accounting and auditing experience in the private sector. Several of them had a number of professional qualifications such as CMA, CPA, CA, CIA, etc, and their professional titles, among others, were ex-chief financial officer, corporate controller, director of internal auditing, SOX implementation specialist, external auditor for small public companies, SOX consultant, and a professor. As instructed, the pilot participants reviewed the survey instrument thoroughly and provided feedback either in writing or through a telephone interview. Based on this collective input, the survey was revised and re-tested with a subset of the original pilot participants. The final survey had a total of 49 questions divided into four separate sections.⁵

A total of 374 surveys responses were received. Of the 374 respondents, about 53 per cent work in a wide variety of finance and accounting-related positions (ie, Chief Financial Officer, Vice President, Controller, Assistant Controller, SOX Specialist, and Accounting Manager) and about 39 per cent practice internal auditing. The remaining 8 per cent were respondents with titles such as internal consultant, financial analyst, compliance director, SOX steering committee member, audit committee chair, president and CEO, risk manager, etc. Additionally, about 75 per cent of the respondents have one or more of the following formal auditing and accounting certifications: CPA, CMA, CA, CISA, and CIA. More than 70 per cent of the respondents have an overall work experience of more than 15 years, with 60 per cent being in the current position anywhere from one to five years. Besides having significant finance and accounting experience, respondents in this sample also spend considerable time on SOX 302/404-related matters. Overall, given the experience level of the survey respondents, combined with the amount of time they spend on managing SOX 302/404 compliance projects, this research study reports the opinions of a very seasoned pool of respondents. It is important to note that there are no external auditors represented in this survey because the objective of this research was to focus only on company-specific experiences in utilising the COSO 1992 Control Framework while evaluating internal control over financial reporting.

Looking at the sample in terms of company size, based on revenue and assets, 374 respondents can be evenly clustered into three major categories: small (less than $500m), medium ($500m to $5bn), and large (more than $5bn) companies, with a slight bias towards fewer respondents from smaller public companies. Further, about 19 per cent of the respondents are from companies with 1,000 or fewer employees. The remaining 81 per cent of the respondents are somewhat evenly divided into two broader groups of companies having employees anywhere from 1,001 to 7,500 and more than 7,500. Although a number of industries are represented in the sample, almost 60 per cent of the respondents in our final sample come from the four industries: manufacturing, financial services, transportation, communication, utilities, and wholesale/retail.

With regard to the SOX 302/404 filing status of the our respondents, 73 per cent of the 374 are from accelerated filer companies, 21 per cent are from non-accelerated filer firms, and about 6 per cent represent foreign filers. Similarly, about 75 per cent of the respondents in our sample are from companies that have already filed their first SOX 302/404 certification, and the remaining 25 per cent are from companies working on filing their first certification.

RESULTS AND DISCUSSION OF THE FINDINGS

To implement Section 404 of the Sarbanes–Oxley Act of 2002, the SEC amended Regulation S-K on 14th August, 2003, to include Item 308. Under these SEC rules, a registrant's annual report must include the following items: ⁶

1. Management's Annual Report on Internal Control over Financial Reporting: This report must contain
   1. A statement of management's responsibility for establishing and maintaining adequate internal control over financial reporting for the registrant;
   2. A statement identifying the framework used by management to evaluate the effective-ness of the registrant's internal control over financial reporting (italics added);
   3. Management's assessment of the effectiveness of the registrant's internal control over financial reporting as of the end of the registrant's most recent fiscal year, including a statement as to whether or not internal control over financial reporting is effective;
   4. A statement that the registered public accounting firm that audited financial statements included in the annual report containing the disclosure required by this item has issued an attestation report on management's assessment of the registrant's internal control over financial reporting.
2. Attestation Report of the Registered Public Accounting Firm: Provide the registered public accounting firm's attestation report on management's assessment of the registrant's internal control over financial reporting in the registrant's annual report containing the disclosure required by this item.
3. Changes in Internal Control Over Financial Reporting: Disclose any changes in the registrant's internal control over financial reporting... that occurred during the registrant's fourth fiscal quarter in case of an annual report that has materially affected, or is reasonably likely to materially affect, the registrant's internal control over financial reporting.

As stated in footnote 3 in accordance with Item 308 (a) (2) (See the italicised portion above), SEC Final Rules for Section 404 and the recently issued Interpretive Guidance for Management recognise the COSO 1992 as a suitable framework for management to conduct its internal control evaluations. Consequently, the COSO 1992 Framework has emerged as the primary control framework for companies of all sizes to assess and report on their internal controls.⁷

Suitability of the COSO 1992 framework per SEC criteria

Neither the SEC Final Rules implementing Section 404 nor the recently released Interpretive Guidance⁸ for management mandate the use of COSO 1992 or any other specific control-evaluation framework to assess the effectiveness of a registrant's internal control over financial reporting under Section 404. The Final Rules, however, do specify the suitability criteria that a framework must meet for it to be considered an acceptable evaluation framework for the purposes of satisfying the requirements under Section 404. According to the Section 404 Final Rules: '...a suitable framework must: be free from bias; permit reasonably consistent qualitative and quantitative measurements of a company's internal control; be sufficiently complete so that those relevant factors that would alter a conclusion about the effectiveness of a company's internal control are not omitted; and be relevant to an evaluation of internal control over financial reporting'. Given that these criteria form the litmus test for an acceptable evaluation standard, it would seem relevant to evaluate COSO 1992 against these criteria. These results are produced in Table 1.

Table 1 - Perceptions about COSO 1992 meeting the SEC criteria of suitability.

Full table

From these results, it is important to note that only about one-third (ranging from 34 to 40 per cent for all four criteria) of our survey participants indicated that COSO 1992, to a large extent, meets the four criteria specified in the SEC Section 404 Final Rules. Similarly, the other one-third of the survey respondents believes that COSO 1992 meets the SEC criteria from no extent to some extent. Almost 10 per cent of the respondents are unable to evaluate the suitability of the COSO 1992 in light of the SEC criteria. Overall, these results are disappointing especially given the fact that almost every single management report under Section 404 claims that the ICFR evaluation was conducted in accordance with the COSO 1992 framework.

From the very early stages of SOX 404 implementation, there continues to be an increased sensitivity on the part of the lawmakers and the SEC regarding the compliance costs being imposed on smaller public companies to comply with this section. Smaller business community and related lobbying groups strongly feel that smaller public companies are being disproportionately impacted by the SOX 404 internal control requirements, and the application of COSO 1992 presents unique challenges to these companies because most of their internal controls are informal and non-formal in nature. To address these concerns, then SEC Chairman William Donaldson appointed a Small Business Advisory Task Force to review a whole host of issues impacting the smaller public companies and Donald Nicolaisen, then SEC chief accountant, asked the COSO committee to develop specifically tailored internal control guidance for the smaller public companies.

Given the emphasis being placed on the needs of the smaller public companies, it is useful to understand whether the perceptions of the smaller public companies in our sample differ in any significant ways from the larger sample on the issue of the suitability of the COSO 1992 Framework. Table 2 presents these results by company size.

Table 2 - Perceptions about COSO 1992 meeting the criteria of suitability by company size.

Full table

The results presented in Table 2 clearly indicate that fewer respondents from smaller companies perceive to a large extent that COSO 1992 meets the four specific criteria as laid out in the SEC Final Rules. In other words, these response statistics suggest that smaller public companies have a less favourable impression of the COSO 1992 Framework than medium to large companies.

Since COSO 1992 is perceived to be 'strongly influenced by the perspective of the independent accountants'⁹ and thus too control-centric, it is plausible that internal auditors in our sample may have a more favourable impression of COSO 1992 when compared with the non-internal auditors (ie, management types). To test for this bias, the sample was subdivided into two sub-groups: internal auditors and management types. These results are presented in Table 3.

Table 3 - Perceptions about COSO 1992 meeting the criteria of suitability by job title.

Full table

As expected, more internal auditors (by a margin of almost 9–14 per cent on criteria #1, #3, and #4, respectively) than the management-types appear to believe that the COSO 1992 Framework meets the SEC criteria to a large extent. This difference of opinion between the two groups disappears, however, when they are asked about criterion #2, which deals with the question of whether COSO 1992 permits, to a large extent, reasonably consistent measurements of a company's internal control over financial reporting (35 per cent of internal auditors versus 33 per cent of management types). This finding is noteworthy. If the underlying control model is unable to produce 'reasonably consistent' conclusions about the effectiveness of a company's controls, tensions are bound to arise between management and auditors on several issues including whether enough control testing has been conducted to provide reasonable assurance.

As criterion #2 is of paramount importance to producing apples-to-apples conclusions on control effectiveness, we further explored this issue by asking the respondents the following two questions:

1. In your opinion, using the COSO 1992 Control Framework, to what extent is it possible to arrive at a reliable pass or fail conclusion on the effectiveness of an entity's system of internal control over financial reporting (ie, one that can be replicated by two independent assurance professionals within a narrow margin of error)?
2. In your opinion, using the COSO 1992 Control Framework, to what extent is it possible to achieve a high level (90 per cent or above) of consensus between company management and their external auditors while opining on the effectiveness of a client's system of internal control under Sections 302/404 when each conducts its assessment on an independent basis?

The rationale to further explore criterion #2 through the above-mentioned two questions is grounded into two main thoughts. First, as the registrants are now required to arrive at a binary (pass/fail) conclusion on the effectiveness of their internal control over financial reporting, it is important that COSO 1992 be able to facilitate such a conclusion to a large extent in a cost-effective way. Secondly, as the current requirements are for the management and the external auditor to separately assess and opine on the effectiveness of a registrant's internal control over financial reporting, it is critical that using the same set of facts, the management as well as the external auditor be able to arrive at a similar conclusion with a much higher degree of consensus. In other words, if a control framework does not lead to a reliable pass or fail conclusion and to a high degree of consensus between managements' and auditors' assessments, then it does not meet the SEC's criterion #2. The responses to the above-mentioned two questions are presented in Tables 4 and 5.

Table 4 - Is it possible to arrive at a reliable pass/fail conclusion on ICFR using COSO 1992?

Full table

Table 5 - Consensus in conclusions between management and external auditor using COSO 1992.

Full table

A review of the results presented in Table 4 indicates that only 22 per cent of the respondents believe that it is possible, to a large extent, to arrive at a reliable pass/fail conclusion on the effectiveness of an entity's internal control over financial reporting. This meagre support is again manifested when we subdivide our sample into smaller public companies (16 per cent) versus medium to large public companies (23 per cent).

Similarly, reviewing Table 5, we find that only 18 per cent of the respondents believe that it is possible to achieve, to a large extent, a high degree of consensus in the managements' and external auditors' assessment and opinion while using COSO 1992. When examined by company size, only 13 per cent of the smaller public company respondents versus 19 per cent of the medium to large public company respondents believe that COSO 1992 results in a high degree of consensus.

Overall, the results presented in Tables 4 and 5 strongly complement and support the findings presented in Table 3. Together, these results raise an interesting question: whether COSO 1992 meets the four criteria as laid out by the SEC in Section 404 Final Rules.

Reliance on COSO 1992 assessment guidance by companies

For the purposes of the issues discussed in this section, it is important to understand what is meant by internal control under COSO 1992: Internal control is a process, affected by an entity's board of directors, management and other personnel, designed to provide reasonable assurance regarding the achievement of objectives in the following categories:

• Effectiveness and efficiency of operations.
• Reliability of financial reporting.
• Compliance with applicable laws and regulations.¹⁰

In other words, the above definition suggests that the COSO 1992 Framework envisions internal control in a much broader sense than the notion of ICFR as indicated in Section 404 and related SEC Final Rules. It is important to note that internal control over financial reporting is subsumed within the above definition under the second objective.

Reviewing the history of the development of COSO 1992 Framework, it appears that there were significant differences of opinions among the various COSO board members at that time. In this regard, Root ⁹ observes that: 'One of the most basic objectives of the Framework was to develop a definition that could serve as the foundation for the balance of the document. This proved to be difficult due to the differing viewpoints that existed among interested parties. There were those who favored a broad definition in recognition of the view that internal control is inclusive of all, or virtually all, management undertakings. Others preferred a narrower definition that focused on internal control over financial reporting. Proponents of the broad concept prevailed. However, commentary was included that explicitly excepted certain management activities from internal control. Exclusions included entity-level objective setting, mission and values statements, strategic planning, risk management, and corrective actions. This compromise was directed at assuaging concerns that a broad definition would increase the risk of misleading external parties regarding management's ability to achieve all objectives associated with a broad definition. Thus, the Framework became influenced by liability considerations surrounding the issuance of reports on internal control to external parties, a largely voluntary practice among large public companies'. When talking about the specific applicability of COSO 1992 for assessing internal control effectiveness, Root¹¹ continues and proclaims that: 'After all, it's no secret that there really is no articulated internal control criteria in the COSO Framework. It provides only a broad definition consisting of three stated objectives, supplemented by a set of five internal control elements. Hence, there [is] ample room for guidance to aid anyone applying the Framework'. In the introduction to the evaluation tools module, the COSO Board states that: 'These evaluation tools are intended to provide guidance and assistance in evaluating internal control systems in relation to criteria for effective internal control set forth in the Framework volume of this report. Accordingly, users of these materials should be familiar with that volume. These tools are presented for purely illustrative purposes. They are not an integral part of the Framework, and their presentation here in no way suggests that all matters addressed in them need to be considered in evaluating an internal control system, or that all such matters must be present in order to conclude that a system is effective. Similarly, there is no suggestion that these tools are a preferred method to conduct and document an evaluation. Because facts and circumstances vary between entities and industries, evaluation methodologies and documentation techniques will also vary. Accordingly, entities may use different evaluation tools, or use other methodologies utilizing different evaluative techniques'.¹² The above caveat in the evaluation tools module that accompanies the Framework supports Root's assertion that the COSO 1992 Framework does not provide specific and sufficient implementation guidance to actually carry out an internal control assessment engagement. This should not be construed to mean that the COSO 1992 Framework should provide a step-by-step or a check-list approach to conducting internal control evaluations. In all fairness, the COSO 1992 Framework should not be faulted for being broad and principles-based because the need of the hour in the 1990s was to consolidate the fragmented thinking on internal control in one place in response to the Treadway Commission's Report on Fraudulent Financial Reporting. Today, however, the demand placed on the COSO 1992 Framework is to provide sufficient implementation guidance that registrants can use to cost-effectively conduct a top-down, risk-based control assessment so that they can legitimately claim that their internal control assessment/evaluation was conducted in accordance with [the COSO 1992's Internal Control-Integrated Framework].¹³

As our review of the internal control certification in the SEC filings of the hundreds of registrants reveals that virtually everyone is claiming that they are conducting their internal control evaluations in accordance with the COSO 1992, to study the validity of this assertion, we asked survey respondents a series of questions to gauge the extent to which COSO 1992 provided these registrants practical or specific guidance while conducting their internal control evaluations. The first question, as stated below, explored respondents' opinions on the level of the specific guidance provided by the COSO 1992 Framework.

• In your opinion, to what extent does the COSO 1992 Control Framework provides specific guidance (as opposed to motherhood and apple-pie type of guidance on elements of an internal control system) to all those who are responsible for assessing and concluding on the effectiveness of a company's system of internal control over financial reporting?

The results for this question are presented in Table 6:

Table 6 - Level of specific implementation guidance provided by COSO 1992.

Full table

A review of these results indicates that only 4 per cent of the survey participants believe that COSO 1992 provides them, to a large extent, with any specific guidance in assessing and concluding the effectiveness of internal control over financial reporting. About 16 per cent even go to the extent of claiming that it does not provide them with any guidance with respect to their internal control evaluation. The majority of the respondents (almost 76 per cent) are willing to give credit to COSO 1992 only to some extent or to a moderate extent.

To further understand the extent to which AS2 (and now AS5) marginalised the broader-level guidance provided in COSO 1992, we asked our respondents whether it is possible for them to arrive at a pass/fail conclusion on the effectiveness of their internal control over financial reporting in the absence of guidance provided in AS2. These results appear in Table 7.

Table 7 - Can COSO 1992 guidance alone lead to a pass/fail conclusion?

Full table

The results presented in Table 7 indicate that only 13 per cent of all respondents believe that, to a large extent, it is possible to arrive at a binary (pass/fail) conclusion using the guidance provided by the COSO 1992 in the absence of AS2. Interestingly, just about the same percentage of respondents (15 per cent) believes that it is not at all possible. Leaving out the ones who are uncertain about the suitability of COSO 1992 to provide such guidance, almost two-thirds of our sample respondents believe that such a pass/fail conclusion is possible only to some extent or to a moderate extent under COSO 1992 guidance.

The distribution of these results does not substantially change when we analyse our sample either by company size or by job title. These findings further reinforce the results reported in Table 6, which concluded that the COSO 1992 Framework provides guidance that is good from a broader perspective but does not provide registrants enough focus in assessing and reporting on internal control to conclude that their assessment was truly carried out in accordance with COSO 1992's Internal Control-Integrated Framework. At this point, it would also be pertinent to mention that when asked 'in your opinion, which one of the following two statements is "more true" for your first-year SOX certification efforts' almost 62 per cent of the respondents chose the statement that the majority of their internal control assessment was largely guided by and conducted in accordance with the PCAOB Auditing Standard #2 as opposed to in accordance with COSO 1992's Internal Control-Integrated Framework.

We also asked our respondents the extent to which their SOX compliance team, at the entity level, evaluated the overall effectiveness of each one of the five main COSO components as part of the process used to form an opinion on the effectiveness of internal control. Only about 35 per cent of the respondents answered this question to a large extent. There was an even split (about 28 per cent each) between the two choices of 'to some extent' and 'to moderate extent'. These responses are consistent with the findings reported above and reinforce the dominance of AS2's guidance in assessing and evaluating internal control over financial reporting.

The following analysis explores the relevance of guidance provided by COSO 1992 in the following four categories: (1) assessment of specific account balances and note disclosures, (2) assessment of fraud risk factors, (3) assessment of IT controls, and (4) mapping of internal control weaknesses to COSO components.

Assessing account balances and note disclosures using COSO 1992

The COSO 1992 Framework describes the five components of internal control as follows:

• Internal control consists of five interrelated components. These are derived from the way management runs a business, and are integrated with the management process. The components are:
• Control Environment— The core of any business is its people — their individual attributes, including integrity, ethical values and competence — and the environment in which they operate. They are the engine that drives the entity and the foundation on which everything rests.
• Risk Assessment— The entity must be aware of and deal with the risks it faces. It must set objectives, integrated with the sales, production, marketing, financial and other activities so that the organisation is operating in concert. It also must establish mechanisms to identify, analyse and manage the related risks.
• Control Activities— Control policies and procedures must be established and executed to help ensure that the actions identified by management as necessary to address risks to achievement of the entity's objectives are effectively carried out.
• Information and Communication— Surrounding these activities are information and communication systems. These enable the entity's people to capture and exchange the information needed to conduct, manage, and control its components.
• Monitoring— The entire process must be monitored and modifications made as necessary. In this way, the system can react dynamically, changing as conditions warrant.¹⁴

To understand the extent to which the survey respondents relied on the guidance provided by each one of the above-mentioned five COSO 1992 components when evaluating internal control over specific account balances, we asked the following question:

• When evaluating internal controls related to most of your specific account balances to what extent did your SOX compliance team specifically rely on the guidance provided by the COSO 1992 Framework for each one of the five COSO components of internal control?

Table 8 presents the answers to this question.

Table 8 - Reliance on five COSO 1992 components to evaluate controls over specific account balances.

Full table

The results suggest that only 23–39 per cent of the respondents believe that one or more of COSO's five elements provided them, to a large extent, with specific guidance while evaluating internal controls related to their company's specific account balances. The control activities element appears to be cited by most respondents (39 per cent) and the risk assessment and information and communication elements are each cited by only 23 per cent of the respondents.

The same question was asked of the respondents again but this time with regard to the note disclosures made in their financial statements. Table 9 summarises these results.

Table 9 - Reliance on five COSO 1992 components to evaluate controls over note disclosures.

Full table

These findings indicate that, across the board, only 16–23 per cent of the respondents believe that their SOX compliance teams relied on the guidance provided by the five COSO components while evaluating internal controls over their company's note disclosures. Almost 10–14 per cent claim no reliance on the five COSO components, with about 15 per cent being uncertain on whether any reliance was placed on the five COSO components. These results do not substantially change when we analyse the responses by company size or job title.

Overall, the results presented in Tables 8 and 9 suggest that a significant majority of the respondents did not use, to a large extent, the guidance provided in five COSO components while evaluating the effectiveness of internal controls over their account balances and related note disclosures. These findings squarely contradict the statements made by the SEC registrants in their public filings that they conducted their internal control assessment in accordance with COSO 1992's Internal Control-Integrated Framework.

Assessing fraud risk vulnerability using COSO 1992

SEC Rules implementing Section 404 and paragraphs 24–26 of the now superceded PCAOB Auditing Standard No. 2 specify management's and external auditor's responsibility for fraud risk and control assessment. In addition to the specific requirements cited in AS2, the external auditor is also required to conduct his/her evaluation of fraud risk controls consistent with SAS 82 'Consideration of Fraud in a Financial Statement Audit'. Collectively, these requirements suggest that management as well as the external auditor must complete an assessment of controls designed to prevent, identify, and detect fraud-related risks that could result in unreliable financial disclosures. SAS 82, Paragraph 16, groups the risk factors that relate to misstatements arising from fraudulent financial reporting into three distinct categories¹⁵:

1. Management's characteristics and influence over the control environment. These pertain to management's abilities, pressures, style, and attitude relating to internal control and the financial reporting process.
2. Industry conditions. These involve the economic and regulatory environment in which the entity operates.
3. Operating characteristics and financial stability. These pertain to the nature and complexity of the entity and its transactions, the entity's financial condition, and its profitability.

These categories suggest that a common-sense approach to assessing fraud vulnerability would start with an assessment of macro-level anti-fraud controls as well as anti-fraud assessment for industry-specific risk factors that would lead to fraudulent financial reporting. Consequently, the survey respondents were asked whether they evaluated macro-level anti-fraud controls and anti-fraud controls for industry-specific risk factors while assessing their internal control over financial reporting. If they answered yes to these questions, they were further probed by asking the extent to which they relied on the guidance provided to them in the five COSO components to carry out these anti-fraud assessments.

Table 10 presents the responses for the fraud assessment at the macro-level as well as for the industry-specific risk factors.

Table 10 - Assessment of fraud risk factors.

Full table

These results present a disturbing picture. Almost 27 per cent of the respondents believe that their SOX compliance team did not complete an anti-fraud risk assessment for industry-specific fraud risk factors as per SAS 82. Similarly, about 31 per cent of the respondents reported that they did not perform a macro-level anti-fraud assessment for fraud risk factors other than for industry specific risk per SAS 82. Remember that the genesis of the financial fraud in scandals (ie, Enron, WorldCom) that gave rise to SOX in the first place was rooted in deteriorating industry conditions.

Table 11 analyses the same responses but this time by company size. What we find is that fewer number of smaller public companies conducted fraud risk assessments, both at the macro-level (68 per cent) as well as for industry-specific risk factors (66 per cent), when compared with the medium to large companies (70–75 per cent, respectively). This is an important finding in light of the fact that 'a 1999 report commissioned by the organisations that sponsored the Treadway Commission found that the incidence of financial fraud was greater in smaller companies'.¹⁶ This finding also calls into question the validity of the arguments advanced by many of the smaller public company lobby groups asking for exemption from Section 404 requirements.

Table 11 - Assessment of fraud risk factors by company size.

Full table

The respondents answering 'yes' to the two fraud-risk assessment questions (see Tables 10 and 11) were further asked about the extent to which they relied on the guidance provided by each one of the five COSO 1992 components in completing such assessments. These results are presented in Tables 12 and 13.

Table 12 - Reliance on COSO 1992 components to assess industry-specific fraud risk factors.

Full table

Table 13 - Reliance on COSO 1992 components to assess macro-level fraud risk factors (other than industry-specific risk factors).

Full table

The results presented in Table 12 indicate that less than 30 per cent (from 20–29 per cent) of the respondents believe that their SOX compliance team relied, to a large extent, on the guidance provided by the five COSO 1992 components when completing their company's anti-fraud assessment for industry-risk factors. Almost 10 per cent of the respondents answered a flat no, indicating that they did not rely on the guidance provided by the five COSO 1992 components and about 15 per cent of the respondents reported being uncertain about whether their SOX compliance team relied on any such guidance while conducting an anti-fraud assessment for their company's industry-specific risk factors.

These results do not improve substantially when respondents are asked the same question but this time with respect to the assessment of the macro-level fraud risk factors. These findings are presented in Table 13.

The results presented in Tables 12 and 13 call into question whether the COSO 1992 Framework 'is sufficiently complete so that those relevant factors that would alter a conclusion about the effectiveness of a company's internal control over financial reporting are not omitted'. In other words, if the COSO 1992 is deemed to provide all the necessary guidance, why are the companies not using it to conduct their fraud risk assessments? Overall, the results presented in Tables 10, 11, 12 and 13 are worrisome because one of the major reasons for passing the SOX legislation was to place increasing focus on the assessment of potential fraud risk factors both by the company managements and their external auditors. It would be unfortunate if this critical goal of SOX was not achieved due to insufficiency of the guidance provided in COSO 1992 or the lack of skill-set in companies and external auditors in applying the COSO 1992 guidance.

Assessing IT controls using COSO 1992

The importance of assessing the effectiveness of general IT controls that support reliable financial disclosures is emphasised repeatedly in the guidance issued by the SEC and the PCAOB. General IT controls relate to: (1) The Information technology control environment, (2) Programme development, (3) Programme change, (4) Access to programmes and data, and (5) Computer operations.

These general IT controls apply to all IT systems including spreadsheet applications that provide information for the 10K and 10Q reports. Controls at third-party service providers that provide services that could impact on the company's external disclosures must also be assessed. Examples of these service providers include offshore software development firms, pension fund administrators, payroll services, software application service providers (ASPs), outsourced procurement, HR activities, and many others. These organisations may or may not be retaining outside experts to conduct SAS 70 Type II or local equivalent internal control reviews (eg, CICA Section 5900 is an example of the Canadian equivalent of a SAS 70 review) to assess and report on the existence and effectiveness of general IT controls and relevant application controls. In some cases, because general IT controls are so critical to the reliability of a company's external disclosures, the SAS 70 reviews that are currently being conducted at third-party sites may not be adequate to meet the SOX 302/404 expectations of external auditors.

Guidance provided by the SEC and PCAOB both indicate that it is important to recognise that companies must evaluate general IT controls that impact on the integrity of the general ledger and accounting systems and systems that store information used to prepare notes to the financial statements. This includes information stored in spreadsheet applications such as MS Excel.

The best guidance currently available to companies to complete general IT control reviews is ISO 17799 Information Technology — Code of Practice for Information Security Management and IT Governance Institute IT Control Objectives for Sarbanes–Oxley. Companies should first identify the full universe of the IT systems, including those controlled by service providers and internal spreadsheet applications that require assessment and then assign responsibility for creating and maintaining these assessments. In addition to assessing general IT controls, companies must also consider IT-related risks that impact all individual financial accounts and note disclosures.

To understand the use and relevance of COSO 1992 and the guidance provided by its five components to the evaluation and assessment of IT controls, the respondents were asked a series of questions. The first question was 'when assessing the effectiveness of your internal controls over IT to comply with SOX 404 requirements, which framework(s) or standard(s) did your organisation use?' The answers are presented in Table 14.

Table 14 - Frameworks used to assess the effectiveness of internal controls over IT.

Full table

The results presented in Table 14 indicate that the CobiT-Control Objectives for Information and Related Technology Framework issued by the IT Governance Institute was most often (about 52 per cent of the time) cited by the survey respondents, followed by COSO 1992 (about 44 per cent of the time). It is important to note that almost 20 per cent of the respondents were uncertain about the framework their company used to assess and evaluate the effectiveness of IT controls over financial reporting. When we analyse the sample responses by company size, we find that the pecking order displayed in Table 14 flips for small companies as they cite more use of the COSO 1992 Framework than the CobiT Framework (56 versus 41 per cent).

The survey respondents were further asked about their reliance on five COSO components to assess and evaluate IT Governance and General IT Controls versus IT Application Controls. Almost 95 per cent of the respondents in the sample reported that their companies evaluated IT Governance and General IT Controls as well as IT application controls. Table 15 presents the related responses.

Table 15 - Use of five COSO 1992 components to evaluate IT governance and general IT controls.

Full table

The results presented in Table 15 indicate that only 17–25 per cent of the respondents believe that their SOX compliance teams relied, to a large extent, on the guidance provided by the five COSO 1992 components. It is also important to note that a significant number (from 34–41 per cent) believe that their SOX compliance teams relied on the COSO 1992 guidance either to no extent or only to some extent while evaluating their company's IT Governance and General IT controls. Similarly, the finding that, on an average, about 15 per cent of the respondents are uncertain whether their SOX compliance teams relied on the COSO 1992 while evaluating IT controls is noteworthy. Almost the same results were found when they were asked a similar question but with respect to IT application controls. Overall, it appears that the SOX compliance teams for our sample companies are relying on COSO 1992 only to a limited extent when it comes to assessing and evaluating IT controls over effective financial reporting.

Mapping control deficiencies to COSO 1992

As discussed earlier, all the regulatory guidance states that the management is required to base its assessment of the effectiveness of internal control over financial on a suitable control evaluation framework and that in the US COSO 1992 meets the PCAOB's and SEC's criteria of a suitable framework. External auditors have, rightfully, interpreted this guidance to mean that 'as part of management's Section 404 assessment, it must document, test, and evaluate the five components of the [COSO] internal control model'.¹⁷ Thus, it would be only logical for a company to map its discovered control deficiencies to the five COSO criteria to determine whether any specific element of the control framework is ineffective to such an extent that the auditor needs to render an adverse opinion on the effectiveness of a company's ICFR.

Pre-survey interviews confirm this thinking as one of the Big 4 public accounting firms asks its clients to map and then aggregate discovered control deficiencies, among other criteria, by five COSO 1992 elements to determine whether any one of these five elements has a material weakness in aggregation and is, therefore, rendered ineffective. Thus, mapping of the discovered control deficiencies, on a practical level, would help the company evaluate whether each COSO component is sufficiently effective so as not to render the overall effectiveness in accordance with the COSO 1992 Framework ineffective. In the absence of such a mapping, we believe that a registrant will have a difficult time, if challenged by the regulatory authorities or in a lawsuit, to demonstrate that they actually evaluated their ICFR in accordance with the COSO 1992 control framework.

To better understand the lack of disclosure in this area by SEC registrants, a series of three questions were asked of the respondents. The first question was 'In your opinion, is it necessary to map all discovered control deficiencies to one or more of the five COSO components to claim that your company conducted its internal control assessment in accordance with the COSO 1992 Framework?' The results are presented in Table 16.

Table 16 - Is it necessary to map control weaknesses to five COSO 1992 components?

Full table

The results presented in Table 16 provide interesting insights into how registrants are interpreting the relationship of discovered control weaknesses with the phrase 'internal control evaluation conducted in accordance with Internal Control-Integrated Framework' [a.k.a. COSO 1992]. Only 13 per cent of the respondents believe that it is absolutely essential to clearly map all discovered control deficiencies to relevant COSO components to legitimately claim that their internal control assessment was conducted in accordance with COSO 1992 (see response #3). A staggering 32 per cent of the respondents believe that it is not even necessary to conduct any such mapping to make the claim that their internal control assessment was conducted in accordance with COSO 1992 (See response #1). About 48 per cent of the respondents adopt a middle-of-the-road position by stating that as long as a company can demonstrate that it evaluated albeit at the entity level all the five COSO components, it is sufficient and reasonable to make a claim in their SEC filings that their internal control assessment was conducted in accordance with COSO's Internal Control-Integrated Framework.

Exploring further, the second question asked 'Did your SOX compliance team map all of the discovered control deficiencies to one or more of the five COSO components as part of the process of forming an opinion on the effectiveness of your organization's internal controls?' The results are presented in Table 17.

Table 17 - How many did the mapping?

Full table

Almost 34 per cent of the respondents reported that their SOX compliance teams did not map the discovered control weaknesses to the five COSO components. Leaving out the 9 per cent who were uncertain about such a mapping, we find that, of the remaining 58 per cent, about 14 per cent mapped only some of the discovered control deficiencies to one or more COSO components, about 24 per cent clearly carried out this mapping, and about 21 per cent proactively mapped all controls (key or non-key) to various COSO components during the documentation phase. This way, if a certain control was found to be inoperative either by design or in operation, the SOX compliance team members would already know which COSO components were impacted.

As mapping of the discovered control weaknesses adds an extra step and additional costs to the SOX 404 compliance process, it would be important to understand the diversity of practice in this area and the basis of conclusion of those companies that claim to have conducted their control assessments in accordance with COSO's Internal Control-Integrated Framework so that others can also learn from their experiences. The last question in this series was asked to understand the usefulness of the guidance provided in COSO 1992 in helping companies map the discovered control deficiencies to one or more of the five COSO components. These results are presented in Table 18.

Table 18 - How useful is COSO 1992 guidance in mapping discovered control deficiencies?

Full table

Only 10 per cent of the respondents indicated that the guidance provided in COSO 1992 was useful to them to a large extent in mapping their control weaknesses to relevant COSO components. About 65 per cent of the respondents believed that the guidance provided by COSO 1992 was useful either to no extent or only to some extent when it came to aggregating the discovered control deficiencies by specific COSO components.

CONCLUSION

The Sarbanes–Oxley Act of 2002 is a landmark piece of legislation that clearly thrusts control governance to the forefront and squarely puts the responsibility for effective internal controls over financial reporting where it truly belongs: the company management. As early as in 1776, Adam Smith, father of modern-day capitalism, in his famous treatise, Inquiry into the Nature and Causes of the Wealth of Nations, wrote '...being the managers of other people's money rather than their own, it cannot well be expected that [managers] should watch over it with the same anxious vigilance with which [they would watch over their own money]'. Holding management responsible and accountable for maintaining effective internal controls over financial reporting would mitigate to some extent the inherent conflict, as identified by Adam Smith. Additionally, as per the requirements of Sections 302 and 404 holding management accountable for maintaining effective internal control system that produces financial disclosures along with faithfully communicating all discovered material weaknesses in this system to their external auditors considerably enhances the quality of an auditors' attestation opinion. Armed with the knowledge about the true state of effectiveness of a company's internal controls over financial reporting, the auditor can now determine the scope of the attestation engagement as well as design and choose appropriate substantive audit tests to opine on the fairness of financial disclosures of a client.

In theory, all this makes perfect sense. The fact that many other industrialised countries and regions with equally developed and sophisticated capital markets (ie, Canada, United Kingdom, European Union, Australia, etc) have considered and consciously made a decision not to go the route of Section 404 internal control certifications confirms that a large majority of these countries believe that the US has not yet gotten the management reporting of internal control over financial reporting right. The SEC Chairman, Christopher Cox, while concluding his opening remarks to the SEC/PCAOB-sponsored Roundtable on Second-Year Experiences with Internal Control Reporting Requirements, noted, 'I hope and expect that today's Roundtable will bring us much closer to the finish line. We have every intention at the SEC and at the PCAOB to get 404 right sooner rather than later'.

Consistent with Chairman Cox's remarks referring to the AS2 that 'no similar guide, however, exists for companies and for their management', one can surmise that a generally accepted control assessment framework for management assessment and reporting on internal control is the pivotal element in ensuring cost-effective compliance with SOX 302/404. This research study highlights the fact that SOX implementation teams across the companies represented in our sample are not overwhelmingly utilising the guidance provided by the COSO 1992 Control Framework to base their internal control assessments. The primary reason for this non-reliance is the principle-based nature of the COSO 1992 Framework that lacks management-centric and risk-based implementation guidance from the perspective of the management.

References

1. This research paper is based on the research monograph titled 'Internal Control: COSO 1992 Control Framework and Management Reporting on Internal Control: Survey and Analysis of Implementation Practices' published by the Institute of Management Accountants in 2006. The complete monograph addresses many more relevant issues related to SOX 404 implementation and is available from the following website:www.imanet.org.
2. See Section II B 3(a) of the SEC Final Rule on 'Management's Reports on Internal Control over Financial Reporting and Certification of Disclosure in Exchange Act Periodic Reports', which states that 'The COSO Framework satisfies our criteria and may be used as an evaluation framework for purposes of management's annual internal control evaluation and disclosure requirements'. Also see footnote 7 to paragraph 5 of the recently issued AS5 that replaces the AS2 and footnote 23 of the recently issued Interpretive Guidance for Management by the SEC.
3. Note that this research paper consists of numerous references to the Auditing Standard No. 2 (AS2), which has now been superceded by the AS5. The subject matter of this research paper is, however, unaffected by the newly issued AS5 because the new auditing standard makes no change with respect to the requirements that ask the auditor to use the same control evaluation framework as that used by the management in its ICFR assessment (see paragraph 5 of the AS5).
4. See, for example five surveys conducted by the Financial Executives International (FEI): May 2003, January 2004, July 2004, March 2005, and April 2006. These surveys are available at www.fei.org.Also see two surveys by NASDAQ, respectively, titled (1) NASDAQ Issuer Survey: Sarbanes–Oxley Act released on 2nd March, 2005 and (2) NASDAQ Issuer Survey: Sarbanes–Oxley Act of 2002 released on 29th September, 2005.
5. The complete survey is available upon request from the author. Please email the author at ppg0@lehigh.edu to ask for a copy of the survey.
6. See Regulation S-K Item 308 (a) (b) and (c).
7. It is, however, important to note that in its Section 404 Final Rule, the SEC makes it clear that 'the final rules do not mandate use of a particular framework, such as the COSO Framework, in recognition of the fact that other evaluation standards exist outside of the United States, and that frameworks other than COSO may be developed within the United States in the future, that satisfy the intent of the statue without diminishing benefits to investors'. Further, in footnote 67, the SEC elaborates on the other evaluation standards by indicating that the Guidance on Assessing Control as published by the Canadian Institute of Chartered Accountants and the Turnbull Report as published by the Institute of Chartered Accountants in England and Wales are examples of other suitable frameworks.
8. US Securities and Exchange Commission 'Commission Guidance Regarding Management's Report on Internal Control Over Financial Reporting Under Section 13(a) or 15(d) of the Securities Exchange Act of 1934.' Effective 27th June, 2007. This guidance is available at the following web address:http://www.sec.gov/rules/interp/200
   7/33-8810.pdf.
9. Root, S. J. (1998). Beyond COSO: Internal Control to Enhance Corporate Governance, John Wiley & Sons, New York, 78.
10. Committee of Sponsoring Organizations of the Treadway Commission (1992). Internal Control — Integrated Framework, Committee of Sponsoring Organizations, Jersey City, NJ, 9.
11. Root, S. J. (1998). op. cit., 117.
12. Committee of Sponsoring Organizations of the Treadway Commission (1992). Internal Control — Integrated Framework: Evaluation Tools, Committee of Sponsoring Organizations of the Treadway Commission, Jersey City, NJ, 1.
13. See Part II.B.2.a of the Section 404 SEC Final Rule. Recall that the SEC Final Rule implementing Section 404 does not name any specific control evaluation framework but clearly states that 'COSO Framework satisfies our criteria and may be used as an evaluation framework for purposes of management's annual internal control evaluation and disclosure requirements'.
14. Committee of Sponsoring Organizations of the Treadway Commission. Internal Control — Integrated Framework: Evaluation Tools, Committee of Sponsoring Organizations of the Treadway Commission, Jersey City, NJ, 13–14.
15. Paragraph 17 in SAS 82 provides numerous examples of risk factors to fraudulent financial reporting in three categories as mentioned in Paragraph 16 in SAS 82.
16. See Section E 'Agency Action to Minimize Effect on Small Entities' and Footnote 190 in the SEC Final Rule on Section 404.
17. 'Sarbanes–Oxley Act Section 404: Practical Guidance for Management' PricewaterhouseCoopers, (July 2004), 26.