A STATISTICAL ANALYSIS OF GEORGIA?S HMA QUALITY ASSURANCE PROCESS Except where reference is made to the work of others, the work described in this thesis is my own or was done in collaboration with my advisory committee. This thesis does not include proprietary or classified information. _____________________________________ James Richard Willis Certificate of Approval: _______________________ __________________________ Rod E. Turochy Frazier Parker, Jr. Assistant Professor Professor Civil Engineering Civil Engineering _______________________ __________________________ David H. Timm Stephen L. McFarland Assistant Professor Acting Dean Civil Engineering Graduate School A STATISTICAL ANALYSIS OF GEORGIA?S HMA QUALITY ASSURANCE PROCESS James Richard Willis A Thesis Submitted to the Graduate Faculty of Auburn University in Partial Fulfillment of the Requirements for the Degree of Master of Science Auburn, Alabama December 16, 2005 iii A STATISTICAL ANALYSIS OF GEORGIA?S HMA QUALITY ASSURANCE PROCESS James Richard Willis Permission is granted to Auburn University to make copies of this thesis at its discretion, upon request of individuals or institutions at their expense. The author reserves all publication rights. ________________________ Signature of Author ________________________ Date of Graduation iv VITA James Richard Willis, son of David Weldon and Dwina Louise (Whittle) Willis, was born on November 27, 1980 in San Antonio, Texas. He graduated from Chester County High School as Salutatorian in 1999. He attended Freed-Hardeman University in Henderson, Tennessee, for three years and then enrolled in Auburn University in August, 2002. In 2003, he graduated summa cum laude and with University Honors with a Bachelor of Science degree in Physical Science from Freed-Hardeman University. In 2004, he graduated summa cum laude from Auburn University with a Bachelor of Civil Engineering. He immediately entered into Graduate School at Auburn University in August, 2004, where he worked under the supervision of the Auburn University Highway Research Center. v THESIS ABSTRACT A STATISTICAL ANALYSIS OF GEORGIA?S HMA QUALITY ASSURANCE PROCESS James Richard Willis Master of Science, December 16, 2005 (B.C.E., Auburn University, 2004) (B.S., Freed-Hardeman University, 2003) 117 Pages Typed Directed by Rod E. Turochy and Frazier Parker, Jr. In the recent years, there has been a push towards allowing contractors to use their own test results in the quality assurance process of Departments of Transportation (DOTs). While this movement saves the states money due to less testing, it has often been wondered if the contractor?s data compare well with GDOT data. An analysis was conducted on hot mix asphalt data for the 2003 construction year in the state of Georgia to evaluate statistically significant differences between the contractor?s and GDOT?s data with the purpose of evaluating Georgia?s QA process and contractor work. It was seen that while the contractor?s means compared well, significnat statistical differences were found with variances. vi ACKNOWLEDGEMENTS The author would like to thank Dr. Rod E. Turochy and Dr. Frazier Parker, Jr., for their guidance and support in the analysis and writing portion of this thesis. The author would also like to acknowledge the National Cooperative Highway Research Program. This research is a small portion of the much larger Project 10-58(02). vii STYLE MANUAL AND SOFTWARE USED Style maunal used: MLA Handbook for Writers of Research Papers (5 th Edition) Computer Software Used: Microsoft Word, Microsoft Excel, Minitab 14 viii TABLE OF CONTENTS LIST OF TABLES?????????????????????????..x LIST OF FIGURES?????????????????????????xiv CHAPTER ONE: INTRODUCTION?????????????????? 1 1.1 Checking the Quality of Pavements?????????????????. 2 1.2 Objectives???????????????????????????.3 1.3 Scope????????????????????????????? 4 CHAPTER TWO: LITERATURE REVIEW??????????????.... 6 2.1 Terms and Definitions??????????????????????..6 2.2 Policy?????????????????????????????10 2.3 Sampling???????????????????????????.. 12 2.4 Quality Assurance Analysis Methods????????????????. 13 2.5 Studies????????????????????????????.. 15 2.6 Summary of Findings??????????????????????... 17 CHAPTER THREE: STATE OF THE PRACTICE????????????.. 19 3.1 2000 University of Kentucky Survey????????????????...19 3.2 2004 Auburn University Survey??????????????????.. 21 3.3 GDOT Practices????????????????????????... 25 3.4 Asphalt Content????????????????????????... 27 3.5 Gradation???????????????????????????. 28 ix 3.6 Summary of Findings??????????????????????... 29 CHAPTER FOUR: METHODOLOGY????????????????? 31 4.1 Data Management???????????????????????? 31 4.2 Overall Data Analysis??????????????????????. 32 4.2.1 The F-test???????????????????????.. 33 4.2.2 The Three t-tests?????????????????????34 4.2.3 Mean Square Deviations??????????????????36 4.2.4 Skewness????????????????????????37 4.3 Individual Project Analysis????????????????????. 38 4.4 Reduced Data Sets???????????????????????... 40 CHAPTER FIVE: RESULTS?????????????????????41 5.1 Overall Project Analysis?????????????????????.. 41 5.2 Project by Project Analysis????????????????????.. 48 5.2.1 Statistical Analysis???????????????????? 48 5.2.2 Precision Tests?????????????????????.. 52 5.3 Reduced Data Set Analysis????????????????????.. 53 5.4 Summary of Findings??????????????????????... 54 CHAPTER SIX: CONCLUSION???????????????????..55 6.1 Overall Data and Reduced Data Set Analyses?????????????. 55 6.2 Project by Project Analysis????????????????????. 57 x 6.4 Final Concerns and Recommendations???????????????.. 59 REFERENCES??????????????????????????.. 61 APPENDICES??????????????????????????... 63 APPENDIX A: OVERALL DATA ANALYSIS SCATTERPLOTS?????... 64 APPENDIX B: SKEWNESS ANALYSIS RESULTS???????????.. 69 APPENDIX C: PROJECT BY PROJECT TABLES???????????? 81 APPENDIX D: PROJECT BY PROJECT ANALYSIS SCATTERPLOTS???. 95 xi LIST OF TABLES Table 2-1: Quality Assurance and Quality Control Properties????????.. 8 Table 2-2: Advantages in Using CPQC?????????????????. 12 Table 2-3: Concerns in Using CPQC??????????????????. 12 Table 2-4: Results from Alabama Study????????????????? 16 Table 2-5: Paired t-Test Comparisons between Kentucky Transportation Cabinet and Contractor Data??????????????????????????.. 17 Table 3-1: Materials Considered for QA/QC??????????????? 20 Table 3-2: DOT Results for Satisfaction????????????????...20 Table 3-3: Contractor Results for Satisfaction??????????????.. 21 Table 3-4: Numerical Survey Averages????????????????? 21 Table 3-5: Property Responsibility??????????????????? 23 Table 3-6: Analysis Methods????????????????????? 24 Table 3-7: Confidence and Satisfaction Table??????????????... 24 Table 3-8: Specification Limits for Independent Samples?????????? 25 Table 3-9: Allowable Percent Difference Between Department and Contractor Acceptance Tests??????????.???????????????. 27 Table 5-1: F-Test Results for Contractor QCT versus GDOT QA??????... 42 Table 5-2: t-Test Results for Contractor QCT versus GDOT QA??????? 42 Table 5-3: F-Test Results for Contractor QCT versus GDOT Comparison???. 43 xii Table 5-4: t-Test Results for Contractor QCT versus GDOT Comparison???. 43 Table 5-5: MSD Results???????????????????????.46 Table 5-6: Skewness Results?????????????????????. 47 Table 5-7: Contractor QCT versus GDOT QA Variance Results by Project??? 49 Table 5-8: Contractor QCT versus GDOT QA Mean Results by Project????.. 49 Table 5-9: Contractor Comparison verses GDOT Comparison Variance Results by Project?????????????????????????????? 49 Table 5-10: Contractor Comparison verses GDOT Comparison Mean Results by Project????????????????????????????... 49 Table 5-11: Asphalt Content Precision?????????????????.. 52 Table 5-12: Gradation Precision???????????????????? 53 Table 5-13: Reduced Data Set Analysis for GDOT QA versus Contractor QCT?. 53 Table 5-14: Reduced Data Set Analysis for GDOT Comparison versus Contractor QCT Comparison?????????????????????. 54 Table 6-1: Sample Sizes for Overall Data Results???????????? 56 Table 6-2: Sample Sizes for Reduced Data Set Results??????????? 56 Table 6-3: Summary of Results for Overall Data Analyses and Reduced Data Set Analyses?????????????????????????????. 56 Table 6-4: QCT versus QA Results Summary??????????????... 58 Table 6-5: Comparison Results Summary????????????????. 59 xiii Table C-1: ?? Sieve Project by Project Table for QCT versus QA??????.. 82 Table C-2: ?? Project by Project Table for Comparison Tests????????. 85 Table C-3: #200 Sieve Project by Project Table for QCT versus QA?????... 86 Table C-4: #200 Sieve Project by Project Table for Comparison Tests????? 89 Table C-5: Asphalt Content Project by Project Table for QCT versus QA???... 91 Table C-6: Asphalt Content Project by Project Table for Comparison Tests??? 94 xiv LIST OF FIGURES Figure 1-1: Variance Occurences in Testing??????????????? 4 Figure 2-1: Quality Assurance Diagram????????????????? 8 Figure 3-1: Terminology Relationships between Georgia and TRB?????? 29 Figure 3-2: Georgia?s Sampling Ratios?????????????????. 30 Figure 4-1: Analysis Thought Diagram?????????????????. 36 Figure 5-1: Mean Comparison from Split Sample Results for Asphalt Content?... 45 Figure 5-2: Skewness QCT-JMF for Asphalt Content???????????.. 47 Figure 5-3: Asphalt Content Variances for GDOT QA versus Contractor QCT?.. 51 Figure 5-4: Asphalt Content Mean Deviations for GDOT QA versus Contractor QCT??????????????????????????????... 51 Figure A-1: Mean Comparison from Split Sample Results for 1? Sieve????... 64 Figure A-2: Mean Comparison from Split Sample Results for ?? Sieve????.. 64 Figure A-3: Mean Comparison from Split Sample Results for ?? Sieve????.. 65 Figure A-4: Mean Comparison from Split Sample Results for 3/8? Sieve????65 Figure A-5: Mean Comparison from Split Sample Results for #4 Sieve????.. 66 Figure A-6: Mean Comparison from Split Sample Results for #8 Sieve????.. 66 Figure A-7: Mean Comparison from Split Sample Results for #50 Sieve???? 67 Figure A-8: Mean Comparison from Split Sample Results for #200 Sieve???.. 67 Figure A-9: Mean Comparison from Split Sample Results for Asphalt Content?.. 68 xv Figure B-1: QCT-JMF Asphalt Content????????????????? 69 Figure B-2: QA-JMF Asphalt Content?????????????????.. 69 Figure B-3: QCT-JMF Asphalt Content (Split Sample)??????????? 69 Figure B-4: DOT-JMF Asphalt Content (Split Sample)??????????? 70 Figure B-5: QCT-JMF #200 Sieve??????????????????? 70 Figure B-6: QA-JMF #200 Sieve???????????????????... 70 Figure B-7: QCT-JMF #200 Sieve (Split Sample)????????????? 71 Figure B-8: DOT-JMF #200 Sieve (Split Sample)????????????? 71 Figure B-9: QCT-JMF #50 Sieve???????????????????.. 71 Figure B-10: QA-JMF #50 Sieve???????????????????.. 72 Figure B-11: QCT-JMF #50 (Split Sample)???????????????.. 72 Figure B-12: DOT-JMF #50 Sieve (Split Sample)????????????? 72 Figure B-13: QCT-JMF #8 Sieve???????????????????.. 73 Figure B-14: QA-JMF #8 Sieve????????????????????. 73 Figure B-15: QCT-JMF #8 Sieve (Split Sample)?????????????.. 73 Figure B-16: DOT-JMF #8 Sieve (Split Sample)?????????????.. 74 Figure B-17: QCT-JMF #4 Sieve???????????????????.. 74 Figure B-18: QA-JMF #4 Sieve????????????????????. 74 Figure B-19: QCT-JMF #4 Sieve (Split Sample)?????????????.. 75 Figure B-20: DOT-JMF #4 Sieve (Split Sample)?????????????.. 75 xvi Figure B-21: QCT-JMF 3/8? Sieve???????????????????75 Figure B-22: QA-JMF 3/8? Sieve???????????????????.. 76 Figure B-23: QCT-JMF 3/8? Sieve (Split Sample)????????????... 76 Figure B-24: DOT-JMF 3/8? Sieve (Split Sample)????????????... 76 Figure B-25: QCT-JMF ?? Sieve???????????????????.. 77 Figure B-26: QA-JMF ?? Sieve???????????????????? 77 Figure B-27: QCT-JMF ?? Sieve (Split Sample)?????????????. 77 Figure B-28: DOT-JMF ?? Sieve (Split Sample)?????????????. 78 Figure B-29: QCT-JMF ?? Sieve???????????????????. 78 Figure B-30: QA-JMF ?? Sieve???????????????????? 78 Figure B-31: QCT-JMF ?? Sieve (Split Sample)?????????????. 79 Figure B-32: DOT-JMF ?? Sieve (Split Sample)?????????????. 79 Figure B-33: QCT-JMF 1? Sieve???????????????????... 79 Figure B-34: QA-JMF 1? Sieve????????????????????. 80 Figure B-35: QCT-JMF 1? Sieve (Split Sample)?????????????.. 80 Figure B-36: DOT-JMF 1? Sieve (Split Sample)?????????????.. 80 Figure D-1: Project by Project Variances for ?? Sieve QCT versus QA????.. 95 Figure D-2: Project by Project Average Means for ?? Sieve QCT versus QA?.....95 Figure D-3: Project by Project Variances for ?? Sieve for Comparison Tests??. 96 Figure D-4: Project by Project Means for ?? Sieve for Comparison Tests???.. 96 xvii Figure D-5: Project by Project Variances for #200 Sieve QCT versus QA???.. 97 Figure D-6: Project by Project Average Means for #200 Sieve QCT versus QA?. 97 Figure D-7: Project by Project Variances for #200 Sieve for Comparison Tests?. 98 Figure D-8: Project by Project Means for #200 Sieve for Comparison Tests??.. 98 Figure D-9: Project by Project Variances for Asphalt Content QCT versus QA?.. 99 Figure D-10: Project by Project Average Means for Asphalt Content QCT versus QA??????????????????????????????.?. 99 Figure D-11: Project by Project Variances for Asphalt Content for Comparison Tests?????????????????????????????..?. 100 Figure D-12: Project by Project Means for Asphalt Content Sieve for Comparison Tests????????????????????????????.??.. 100 1 CHAPTER ONE INTRODUCTION When it comes to buying products, almost any consumer wants to get exactly what he or she ordered. What would happen if a person were to visit a restaurant and ordered a sandwich without mayonnaise, but the bread was tarnished by only a trace of the unwanted condiment? Most would not accept the sandwich and require another one to be produced, preferably at some reduced cost. What would a person do if they had ordered clothes online and upon arrival found a defect in one of the shirts? That individual would most likely call the company and ask for another to be shipped to replace the faulty shirt. The idea of buying quality merchandise is nothing new to the world of personal consumerism; however, it is also something that most Departments of Transportation (DOTs) also hold in high regard for their pavements. DOTs, like consumers, want what they ordered, and if the contractor does not provide it within a specified set of limits, DOTs sometimes want new products. Other times, the DOTs request reduced prices on the inferior products because the life cycle costs of the delivered product will be greater than that of the requested product due to more frequent maintenance and the possibility of a quicker replacement. 2 1.1 Checking the Quality of Pavements Quality assurance is the testing process whereby a material is said to meet the specifications set forth for it. It has long been the practice of DOTs to conduct specific tests, depending on the individual department, to determine if the material the contractor was using to construct the roadway in question met the specifications in the job-mix formula (JMF). However, in the recent years, the DOTs have been given the option of relieving themselves of that burden and placing it on the shoulders of the contractors. In 1995, a federal regulation entitled 23 CFR 637, Part B (Quality Assurance Procedures for Construction [QAPC] was enacted that allowed DOTs to begin using their contractor?s test results for acceptance. In order to help ease the transition between testing agencies, the American Association of State Highway and Transportation Officials (AASHTO) published the ?Implementation Manual for Quality Assurance? and ?Quality Assurance Guide Specification? in the year 1996. Some states have embraced the idea of using contractor?s tests for material acceptance because it eases the workload for the DOTs. Not only would time be saved, but also the expenses of running tests would be minimized if the contractors were responsible. However, while some states have embraced the idea whole-heartedly, other states have been slightly more hesitant to turn over acceptance testing to contractors. In order to investigate this concern, the National Cooperative Highway Research Program launched project 10-58 (02), Using Contractor-Performed Tests in Quality Assurance. The objective of the project ?is to develop procedures to assist state DOTs in effectively using contractor-performed tests in the quality assurance process.? To accomplish this goal, the results of different DOTs using contractor-performed quality 3 tests in the quality assurance process needed to be studied. The methods used by each state were surveyed so that other suitable possible sources of data might be found to undergo analyzation for the stated objective, and those states that seemed to fit were asked to send data to be analyzed. Georgia was the first state to respond to the request. While Georgia does not use contractor data in its acceptance of concrete products, it does use contractor-performed tests on hot-mix asphalt (HMA) projects. Both the contractor and DOTs test for the pavement?s asphalt content and percent passing the 1.5?, 1?, ??, ??, 3/8?, #4, #8, #50, and #200 sieves. It should, however, be noted that the Georgia DOT tests for acceptance of mat density and smoothness. 1.2 Objectives The objective of this project was to numerically investigate the quality assurance process in the state of Georgia by the use of statistical measures and numerical comparisons. Specifically, data were analyzed to determine if sets of results provided by the contractor differed significantly from those provided by the DOT. Differences were evaluated statistically by F-tests for variance, t-tests for means, skewness, and mean square deviations for accuracy and precision. It is not the objective of this report to prove or disprove data manipulation occurs on the part of either the contractor or the DOT; however, data manipulation could be a possible reason if contractor and DOT results are shown to have varied significantly from each other. 4 1.3 Scope The analysis process was conducted on data collected during the year 2003 for Georgia?s HMA projects. Tests were performed on both independent and split samples; however, while the independent samples might show differences in material uniformity, the split sample results analyze the differences in testing methods. Figure 1-1 visually shows where variances can arise during the material testing process, and it also shows which variances are tested by independent and split samples. Figure 1-1. Variance Occurrences in Testing. Three different data sets were analyzed: the overall data set, a reduced data set, and project data sets. The overall analysis was conducted on all the data provided for the 2003 construction season. The nine properties previously mentioned were subjected to F tests, t-tests, mean square deviations, and skewness analyses to determine if the values the contractors were returning to GDOT were greatly different than those GDOT was finding from their own tests. The reduced data set was compiled by combining all the data from projects that had at least six GDOT test results. A large project was defined as having at least six tests for the purposes of this report. F and t-tests were used on the 1/2? sieve, #200 sieve, and Independent Split ? 2 total = ? 2 sampling + ? 2 testing + ? 2 material ? 2 total ? 2 testing (theoretically) 5 asphalt content data to see if many of the smaller projects might be skewing the results of the overall project analysis. The final type of data analyzed was individual project data. Data from all projects that had at least six GDOT test results were subjected to F and t-tests for the 3/8? sieve, #200 sieve, and asphalt content. These tests were conducted to determine how many projects had significant differences in both variances and means of GDOT and contractor test results. These studies strengthened observations of tendencies, such as GDOT?s variances being larger than the contractor?s, that might consistently occur. 6 CHAPTER TWO LITERATURE REVIEW Since the policy change allowing the use of contractor data for acceptance was codified in the Code of Federal Regulations (CFR), 23 CFR 637, Part B (Quality Assurance Procedures for Construction [QAPC]), state departments of transportation (DOTs) have begun to use contractor data for decisions regarding payment and acceptance. However, since the implementation of this policy, few states have investigated whether the states and contractors have adequately adapted to the change in responsibilities. The slow transition from policy to practice could be due to the fact that each state is responsible for determining what might be appropriate for its own practice instead of a standard being issued by a national authority. Other factors that might have influenced the lack of study could be reluctance on the part of the DOT to trust the contractor to adequately report the test results or the lack of understanding the terms and concepts involved in the process of verifying the quality of a material. 2.1 Terms and Definitions If one looks through specification manuals or asks engineers across the country to define the terms related to the quality control/quality assurance process, the answers would be varied and inconsistent despite the efforts of organizations such as the Federal Highway Administration (FHWA) and the Transportation Research Board (TRB). In 7 1995, when the policy change occurred in the CFR, the FHWA created a list of definitions of terms pertaining to the quality control/quality assurance process; however, to further stress the point, the TRB created the Glossary of Highway Quality Assurance Terms in April of 2002. In this document, the original definitions were further refined to more adequately describe what the terms had come to mean. Quality assurance is the first and most important term defined in either of the documents. The original FHWA definition stated that QA was ?all those planned and systematic actions necessary to provide confidence that a product or service will satisfy the requirements for quality? (FHWA, 1995). When the TRB defined this term in its circular, it refined the definition to the following: ?All those planned and systematic actions necessary to provide confidence that a product or facility will perform satisfactorily in service. QA addresses the overall problem of obtaining the quality of a service, product, or facility in the most efficient, economical, and satisfactory manner possible? (TRB, 2002). The term quality control (QC) is often erroneously interchanged with that of quality assurance. The FHWA defined the term as ?all contractor/vendor operational techniques and activities performed or conducted to fulfill the contract requirements? (FHWA, 1995); however, QC should be viewed more in the lines of process control. The TRB defined it as ?those QA actions and considerations necessary to assess and adjust production and construction processes so as to control the level of quality being produced in the end product? (TRB, 2002). To further alleviate confusion between the terms, the Glossary of Highway Quality Assurance Terms included Figure 1 and Table 1 which have been recreated as 8 Figure 2-1 and Table 2-1 reproduced herein. As can be seen, along with the many other differences, it should be noted that contractors are responsible for QC practices while the DOT is responsible for the QA process. Table 2-1. Quality Assurance and Quality Control Properties (TRB, 2002). Quality Assurance (QA) Quality Control (QC) ? Making sure the quality of a product is what it should be ? Making the quality of a product what it should be ? Highway agency is responsible ? Producer/contractor is responsible ? Includes QC ? A part of QA ? Doing the right thing ? Doing things right ? Motivates good QC practices ? Motivated by QA and acceptance procedures Figure 2-1. Quality Assurance Diagram (TRB, 2002). As seen in Figure 2-1, there are two other legs to the QA process: acceptance and independent assurance. Acceptance is simply ?the sampling and testing, or inspection, to determine the degree of compliance with contract requirements? (TRB, 2002). This Quality Assurance in Construction Process Control (Quality Control) Acceptance Independent Assurance 9 definition is similar to the original quality control definition provided by the FHWA; however, the latest definitions show that acceptance relates to the contract requirements while QC is related to the creation of the contracted product. The other process, besides acceptance and QC, is the independent assurance (IA) testing process. By definition, this type of testing is done by a third party, though some states? practices include IA tests conducted by DOT staff. IA is defined as ?a management tool that requires a third party, not directly responsible for the control or acceptance, to provide an independent assessment of the product and/or the reliability of the test results obtained from the process control and acceptance testing? (TRB, 2002). The results of these tests are not to be used in determining if the product should be accepted or not. The FHWA simplistically defined verification as that process which was used to validate the quality of a product, and the newer TRB definition did not add much to the previous. Verification is ?the process of determining or testing the truth or accuracy of test results by examining the data and/or providing objective evidence? (TRB, 2002). One important thing to know about verification testing is that it can be done during multiple phases of the quality assurance process. Verification could be incorporated into the IA process, and this would be used to help verify the results of either the contractor?s QC tests or the agency?s acceptance tests. The acceptance program could also be a viable home for a verification program ?to verify contractor testing used in the agency?s acceptance decision? (TRB, 2002). Another term pertinent to understanding the QA/QC process is pay factor or pay adjustment. The TRB says that pay factors are percentages used to raise or reduce the 10 contractor?s payment based upon the test results that estimate the quality of the product. In most cases, pay factors are attributed to the product on a characteristic by characteristic basis. For example, a pavement might exceed the requirements for density inducing a pay factor over 100%; however, the mix might not be exceptional in its gradation causing the pay factor for the gradation to be below 100%. Many agencies determine which characteristics to include in their payment process, and they split the percentage of the pay in some way across the required characteristics. Taking the example above, the DOT might decide 60% of the pay should be based upon density while only 40% is based upon the mix?s gradation. The two percentages are then multiplied by their respective pay factor, and they are then added together to produce a composite pay factor. Another common approach is to take the lowest of the pay factors among all the properties and use that as the overall pay factor. 2.2 Policy The FHWA created requirements for process control and acceptance programs. The acceptance process can include QC testing results when the following requirements are met (FHWA, 1995): 1. The labs and personnel involved in the testing process are qualified. 2. Independent samples must be used to validate the quality of the material in question. 3. An independent assurance program must be in place. 11 The FHWA also produced a list of requirements for the acceptance programs including the following three items (FHWA, 1995): 1. A frequency guide schedule 2. Identification of sampling location 3. Identification of attributes to be investigated. In order to accurately set up policies and specifications for using contractor- performed quality control (CPQC) tests, a set of objectives needed to be explicitly outlined. In a report submitted to the Kentucky Transportation Cabinet (KYTC), four distinct objectives for CPQC testing were defined (Mahboub et al., 2001). 1. Improve the quality of the materials and processes used in the construction of highway projects, and reduce the life cycle costs for the facilities involved. 2. Redirect the responsibility for quality control on projects to the contractor. 3. Reduce the disputes between the DOT and its contractors. 4. Enhance the construction schedule and the Department?s effort on quality management. If these objectives and policies are followed, then the use of CPQC testing should hold advantages for both the contractor and the DOT; however, this is not a system that does not raise any concerns among its users. Tables 2-2 and 2-3 provide lists of advantages and concerns in the CPQC process. These lists were compiled from surveys sent out in a 2001 study for the KYTC. 12 Table 2-2. Advantages in Using CPQC (Hancher et al., 2002). Agency (DOT) Contractor ? Contractor responsible for their own products ? Reduction of state personnel ? Gaining knowledge by contractors ? Improving dispute resolution ? Quality improvement ? Contractor more suitable for control ? Improving schedule ? Improving quality ? Better dispute resolution Table 2-3. Concerns in Using CPQC (Hancher et al., 2002). Agency (DOT) Contractor ? Validity of test data ? Insufficient certified technician pool ? Insufficient QA ? Lack of training ? DOT losing expertise ? Contractor operating at lower end of specification ? Fear of losing control on project ? Lack of understanding ? Capability of technicians and facilities ? Cost of QC ? Lack of trust ? Lack of training ? Honesty of some contractors ? Expensive independent test agencies ? Different goals of contractor and DOTs The agency also hopes that giving this added responsibility to the contractor will increase the importance of quality in the contractor?s mind (Hancher et al., 2001). 2.3 Sampling During the sampling process, three different types of variations can be investigated: materials, sampling, and testing. While the FHWA policy requires ?all samples used for quality control and verification sampling and testing? (FHWA, 1995) to be random samples, the DOT is responsible for deciding whether it shall use independent or split samples. Each sample type has intrinsic characteristics that determine its appropriate uses. 13 Independent samples are, according to the Glossary of Highway Quality Assurance Terms, ?taken without regard to any other sample that may also have been taken to represent the material in question? (TRB, 2002). These samples are taken at separate times, locations, and even possibly volumes. In turn, when independent samples are tested, they have the ability to provide data on the variabilities of all three parameters (materials, sampling, and testing) (Schmitt, 2001). Split samples, unlike independent samples, come from one material source. One sample is taken, and then it is broken into portions to give to the laboratories running the inquiries (TRB, 2002). Due to the nature of this sampling procedure, the only variability should come from the testing procedures and technicians in the different laboratories. This is due to the samples coming from the exact same location; the materials, production, and sampling should all be the same (Schmitt, 2001). A study conducted at the University of Wisconsin-Madison further investigated differences that might occur in split samples versus independent samples. A sample of 16 projects across six states was analyzed statistically, and the variances of the split samples were compared to the variances of the independent samples. As expected, the results showed that the independent samples had greater variances than the split samples 96.2% of the time (Schmitt, 2001). 2.4 Quality Assurance Analysis Methods Once the samples have been taken, the test results must be analyzed by using one of many possible methods. While there are a variety of possibilities to choose from for the type of analysis to be used, two are becoming the most popular choices for state 14 DOTs. The percent within limits (PWL) method is a way to compare tests results to specifications. The second method, statistical analyses using F and t-tests, is a way to compare two test results to each other. The PWL method is now highly supported by the FHWA. In this method, the state DOT must decide on upper and lower limits for the chosen characteristics. These limits are normally set up as two standard deviations in each direction from the mean using the normal distribution function. These limits can either be chosen based upon past data or experience in the field. The test results are then plotted, and the percent of test results within the two limits is calculated. Payment and acceptance is based upon how close the test result averages are to the target and variability (Sholar et al., 2004). F and t-tests are another form of statistical analysis that are commonly used to help analyze the consistency of contractor and DOT data. While the theory behind these tests will be explained in greater detail at a later time, the purpose of each test should be noted. F-tests are used to determine if the variabilities of two data sets can be shown to be different to a specified level of significance. This test must be performed first to determine the appropriate t-test. If the F-test shows the variabilities are the same, then a t-test assuming equal variances is conducted. If the variabilities are statistically different, then a t-test using unequal variances is used. T-tests are used to the test statistical significance of differences between the means of two data sets (Mahboub et al., 2004). If the test results come from split samples, then a different type of t-test should be conducted. Since the same material is being tested, the variabilities should be the same. The paired t-test is used on split samples because it allows a one-to-one comparison and is more powerfully statistically (Mahboub et al., 2004). 15 While both of these statistical analyses are both commonly and easily incorporated into the QA programs in many states, they are not perfect, and two major concerns can be voiced. The first concern is the lack of data that can be analyzed. While large projects have an abundance of quality assurance testing conducted on the project, smaller projects might only have three or four tests. It is difficult, if just a few tests are available, to accurately know whether or not the assumptions needed to conduct a proper t-test have been fulfilled. The more degrees of freedom the particular test has, the more accurate its depiction of mean and variability similarity will be (Hancher et al., 2002). The second concern arises from the first concern. If there is a lack of data, it is difficult to prove normality, the test results following the normal distribution. While testing and studies have shown that constructor materials test data tend to conform to a normal distribution, both the percent within limits method and F and t-tests rely on data normality (Hall et al., 2002). If the data for some reason are not normal, then the tests might not accurately portray the statistical differences between the data sets (Hancher et al., 2002). 2.5 Studies A study was conducted by the Highway Research Center at Auburn University that compared contractor?s data to that of the Alabama DOT for a sampling of projects for the years 1990, 1991, and 1992. It should be noted that this study was conducted before the change in the Code of Federal Regulations, but the findings are still applicable to this study. When the team conducting the research specifically targeted the asphalt content?s difference from the job-mix formula (JMF), they found that means of contractor 16 and DOT measurements were different for about 1/3 of the mixes. For the first two years, neither the state DOT nor the contractor?s measurements were consistently further from the target than the other. However, in 1992, there were significant differences in means for 17 out of 48 mixes. The state DOT?s test results consistently showed more deviation from the target, having the larger means for 15 of the possible 17 mixes. General trends observed were that the variability decreased and the accuracy increased with time (Parker et al, 1995). After the CFR changed, another project was sponsored by ALDOT to statistically analyze measurements of HMA properties as the Superpave mix design system was implemented. The team studied data from 1997 to 2000 for three properties: asphalt content, air voids, and mat density. Using a 5% significance level on F and t-tests, contractor and DOT data were analyzed. Table 2-4 provides the results which show that more statistical differences occurred with variances than means. Table 2-4. Results from Alabama Study (Parker et al., 2002). Statistical Difference @ 5% Variability Means Asphalt Content 1997 Yes Yes 1998 Yes No 1999 Yes No 2000 Yes No Combined Yes No Air Voids 1997 No No 1998 Yes No 1999 Yes Yes 2000 Yes No Combined Yes No Mat Density 1997 No No 1998 Yes Yes 1999 Yes Yes 2000 Yes Yes Combined Yes Yes 17 The University of Kentucky conducted a statistical analysis of the Kentucky Transportation Cabinet?s (KYTC) verification data versus the contractors? data. Means were compared using paired t-tests for both hot-mix asphalt (HMA) and Portland cement concrete (PCC). Results, as shown in Table 2-5, indicate that means of KYTC?s data set are comparable to the means of contractors? data. Similar to the Alabama?s studies, the contractors? measurements consistently had smaller standard deviations. This was explained by the possibility of using a feedback loop to correct variabilities within the project (Mahboub et al., 2004). If this is truly the reasoning behind smaller contractor variabilities, then the contractors are doing what should be done, and the DOTs should not be concerned with transferring responsibilities in the QA process to them. Table 2-5. Paired t-Test Comparisons between Kentucky Transportation Cabinet and Contractor Data (Mahboub et al, 2004). Significant difference at 5% KY testing category Mean (p-value) Standard Deviation (p-value) HMA-air No (0.462) Yes (<0.0001) HMA-asphalt content No (0.851) Yes (<0.0001) HMA-VMA No (0.83) No (0.854) PCCP-air No (0.823) Yes (0.004) PCCP-slump No (0.822) Yes (<0.0001) PCCP-strength No (0.854) Yes (0.002) PCC-structural-air No (0.766) No (0.219) PCC-structural-slump No (0.680) No (0.669) PCC-structural-strength No (0.480) No (0.223) 2.6 Summary of Findings In 1995, states were allowed to begin using contractor test results in the acceptance process with the implementation of 23 CFR 637 Part B. This was done to help the state transportation agencies save both time and money in the testing phases of projects. Since that time, studies have been conducted in Alabama and Kentucky to see how contractor test results compare to those of the DOT. The 1992 Alabama study 18 showed that the mean asphalt contents of the contractor and DOTs were different for about 1/3 of the mixes tested; however, the later Alabama study and the Kentucky study both showed that statistically significant differences in variances were more likely to be found than in means. All three studies found contractors to have smaller variances and mean differences from the target. 19 CHAPTER THREE STATE OF THE PRACTICE The QA process has relatively few policy requirements in terms of how it should be conducted. The freedom is given to the state DOT to come up with an appropriate set of regulations for its contractors to follow as long as they conform to 23 CFR 637 Part B. The following are just a few of the questions each state must answer: ? What properties should be considered in the QA process? ? How many times should each property be tested for by each organization? ? Who should conduct the tests? ? What will the role of the contractor be? ? What methods should be used to test each property? ? Once the test results are in, how should it be decided if they are adequate for acceptance? With these questions and a host of others to be answered by each state, it is not surprising to see that possibly no two states conduct the QA process in the exact same way. In fact, the diversity seen from state to state is quite staggering. 3.1 2000 University of Kentucky Survey In 2000, a group from the Kentucky Transportation Center at the University of Kentucky at Lexington sent surveys state DOTs and contractors. The survey consisted 20 of questions in regards to which programs were implementing contractor data as a viable option for product acceptance. Another area of concern was the satisfaction with the current QA practices. Thirty state DOTs and 12 contractors responded to the survey. Table 3-1 is an overview of the four most common products included by DOTs in their 2000 QA/QC specifications. As is evident from the responses, HMA is the material for which contractor data is most widely used for QA purposes (Hancher et al., 2002). Table 3-1. Materials Considered for QA/QC. Materials Percentage Using QA/QC Grading/Earthwork 26.7% PCCP 50.0% HMA 86.7% Concrete Bridge Deck 50% Tables 3-2 and 3-3 give summaries of the responses the DOTs and contractors, respectively. Each DOT and contractor surveyed was given the chance to voice its opinion of the effects that contractor-performed acceptance testing was having in four categories where it was supposed to bring about positive influence: project quality, overall project cost, project schedule, and project disputes. The results of this survey were given in number form with a 5 representing very positive and 1 representing very negative; however, some agencies chose not to answer, and those agencies had their responses marked as unidentified. Table 3-2. DOT Results. Satisfaction Rating Project Quality Overall Project Cost Project Schedule Project Disputes Very Negative 3.4% 6.9% 0% 3.6% Negative 0% 17.2% 3.4% 7.1% No Effect 6.9% 34.5% 69.0% 21.4% Positive 51.7% 13.7% 3.4% 42.9% Very Positive 13.7% 0% 0% 0% Unidentified 24.1% 27.7% 24.1% 25% 21 Table 3-3. Contractor Results. Satisfaction Rating Project Quality Overall Project Cost Project Schedule Project Disputes Very Negative 0% 16.7% 0% 8.3% Negative 8.3% 25% 0% 16.7% No Effect 41.7% 8.3% 33.3% 33.3% Positive 25% 41.7% 50% 0% Very Positive 25% 8.3% 16.7% 41.7% When the averages were computed, it was found that the contractors felt more positive about using their test results in the categories of Overall Project Cost, Project Schedule, and Project Disputes. These results are in Table 3-4. Project Quality is the only area where the state DOTs felt more confident. This seems odd that the DOT would feel more positive about using the contractor data than the contractors. Table 3-4. Numerical Survey Averages (Hancher et al. 2002). Category DOT Averages Contractor Averages Project Quality 3.95 3.67 Overall Project Cost 2.76 3.00 Project Schedule 3.0 3.83 Project Disputes 3.38 3.50 3.2 2004 Auburn University Survey In 2004, Auburn University?s Highway Research Center began thoroughly looking at the QA/QC practices of different Departments of Transportation for Project 10-58(02) for the National Cooperative Highway Research Program. A survey was sent to 25 different Departments of Transportation to investigate their quality assurance practices. This survey was similar to the survey sent by the University of Kentucky, but questions were much more specific regarding details of the actual QA practices. 22 Fourteen states and Federal Highway Administration?s Western Lands Office responded for hot mix asphalt. Table 3-5 summarizes which organization has testing responsibilities for HMA properties. The number in each column represents how many agencies responded use what type of testing for the specified property. This table shows that while some states look at similar material properties; many states choose to test more obscure properties. The four properties that are most consistently tested by both the contractor and the agency are gradation, asphalt content, mat density, and voids in mineral aggregate. While there may be some consistency in material properties used, the testing methods for a particular property may be different. Lot sizes vary from 500 tons maximum tonnage to one day?s work. The contractors sometimes will use split samples instead of using independent samples. 23 Table 3-5. Property Responsibility. Property Contractor Agency Both None Gradation 2 0 11 2 Asphalt Content 1 1 13 0 Voids in the Mix 0 1 7 7 Voids in Mineral Aggregate 1 2 9 3 Voids Filled with Asphalt 1 0 2 12 Marshall Stability 0 0 2 13 Flow 0 Moisture Content 1 1 6 7 Layer Thickness 1 4 1 9 Mat Density 0 3 12 0 Smoothness 3 5 6 1 Hveem Properties 0 2 2 11 Boil Test 0 1 0 14 Abson Recovery 0 1 0 14 Maximum Gravity 0 0 1 14 Dust to Asphalt 0 0 2 13 Retained Tensile Strength 0 0 1 14 Fine Aggregate Angularity 0 0 1 14 Clay Content 0 0 1 14 Lottman 0 14 Joint Density 0 0 2 13 Lime Gradation 0 0 2 13 Sand Equivalency 0 0 1 14 Air Voids 0 0 1 14 Mat Temperature 0 1 0 14 Each agency determines the most appropriate way to decide if inconsistencies occur between the contractors? test results and their own. Table 3-6 shows the variety of analysis methods for the four most common tests performed by both the contractor and the agency. More responses would have made for a more accurate representation; however, the variety can still be seen from the small sampling. 24 Table 3-6. Analysis Methods. Property Numerical Criteria F and t test t test only Other Gradation (4 responses) 25% 50% 25% 0% Asphalt Content (5 responses) 20% 60% 20% 0% VMA (3 responses) 33.3% 33.3% 0% 33.3% Mat Density (4 responses) 25% 25% 25% 25% Like the survey conducted by the University of Kentucky, the Auburn survey asked the DOTs if they are confident the contractor?s test results provide the same control of quality as their test results. This was asked for each of the properties, and then an overall satisfaction of the program question was posed. Table 3-7 summarizes survey responses. Only the overall and top four properties results are summarized in the table. The results show that, overall, the DOTs feel mostly confident using contractor test results in the quality assurance process. Table 3-7. Confidence and Satisfaction Table. Property Confident Mostly Confident Neutral Not Totally Confident Not Confident Gradation 27.3% 36.4% 27.3% 9.1% 0% Asphalt Content 20% 60% 10% 10% 0% VMA 14.3% 71.4% 14.3% 0% 0% Mat Density 44.4% 33.3% 22.2% 0% 0% Satisfaction Satisfied Mostly Satisfied Neutral Not Too Satisfied Not Satisfied Overall 33.3% 26.7% 33.3% 6.7% 0% 25 3.3 GDOT Practices The Georgia DOT accepts HMA based on four material properties: asphalt content, gradation, mat density, and smoothness. However, contractors? data is used only for asphalt content and gradation. Georgia DOT tests for both mat density and smoothness are used for acceptance. The Georgia DOT?s quality assurance process terminology is different from the FHWA and TRB definitions provided in Chapter 2. GDOT uses three different terms to designate testing methods and who did the testing. QCT represents the regular contractor testing that is required, and this is related to the TRB term quality control. Georgia requires one test for every 500 ton sublot, and a LOT is equal to one day?s production (GDOT, 2005). The abbreviation QA stands for the Georgia DOT?s testing that is compared to the QCT test results which falls under the TRB definition of acceptance. These tests are conducted twice for every 5 lots or 5 days, whichever is less. The QA tests are conducted on independent samples, and the results are compared to the JMF using the set of specification limits shown in Table 3-8. While there are no specification limits set, the percents passing the 1?, 0.75?, and #50 sieves are determined (GDOT, 2005). Table 3-8. Specification Limits for Independent Samples (GDOT, 2005). Property +/- Specification Limit Asphalt Content 0.4% 0.5? Sieve 6.0% 0.375? Sieve 5.6% #4 Sieve 5.6% #8 Sieve 4/6% #200 Sieve 2.0% 26 If these specification limits are met with the QA test results, then the QCT tests for asphalt content and gradation are permitted for use in acceptance and the calculation of the pay factors. Pay factors are calculated for asphalt content, designated sieve sizes, and mat density. The maximum pay factor is 1; therefore, no bonuses are given for exceptional work. The third type of testing is DOT comparison tests. These are on split samples with contractor QCT samples. The QCT and DOT Comparison tests fall under the TRB term independent assurance. These split samples (Contractor QCT and DOT comparison) are taken once for every 10 lots, and the results are compared one to one with criteria in Table 3-9. The purpose of the DOT comparison test is to verify the QCT results. As with the QCT and QA samples, the percents passing the 1?, 0.75?, and #50 sieves are determined (GDOT, 2005). 27 Table 3-9. Allowable Percent Difference Between Department and Contractor Acceptance Tests (GDOT, 2005). Property Surface Mixes Subsurface Mixes Asphalt Content +/- 0.4% +/-0.5% 0.5? Sieve N/A +/-4% 0.375? Sieve +/-3.5% +/-4% #4 Sieve +/-3.5% +/-3.5% #8 Sieve +/-2.5% +/-3.0% #200 Sieve +/-2.0% +/-2.0% If the DOT comparison and QCT test results compare favorably and if the DOT QA test results meet the specification mix requirements, QCT test results are used for pay factor computation. If these conditions are not met, additional sampling and testing is conducted to resolve the differences. If the differences cannot be resolved, the QCT test results may be replaced with the GDOT test results for pay factor computation. 3.4 Asphalt Content GDOT allows asphalt content to be tested by either extraction or by the ignition oven as specified in its specification manual. GDT 83 is designated as the Method of Test for Extraction of Bitumen from Paving Mixtures using the Vacuum Extractor. This test method uses a solvent and a vacuum to remove the bitumen from the HMA sample. The percent asphalt is calculated by then subtracting the remaining weight of the sample from the original weight as a percentage of the original weight (GDOT, 2005). The Method of Test for Determining AC Content by Ignition Oven, GDT 125, is a much simpler way of determining the asphalt content of a sample. A sample of the HMA is placed in an oven set to 1000 o F. Inside the oven is a balance that measures the weight of the sample. The HMA should remain in the oven until the balance stabilizes on a 28 weight. The percent asphalt is then calculated in the same way as for the extraction method (GDOT, 2005). One reason asphalt content might have been chosen to undergo the QA process is its correlation to performance. In 2004, a study conducted at North Carolina State University documented testing that had been conducted to characterize differences in a pavement?s fatigue life and initial stiffness based upon its material properties. In testing asphalt content, a set of general mixes and a set of North Carolina DOT-specific mixes were created with asphalt content at its optimum level and at -0.5% of optimum. On the North Carolina DOT mixes, the reduction in asphalt content reduced the fatigue life of the pavement by 18-25%; however, the general mixes showed a reduction of fatigue life of up to 50%. The initial stiffness did not seem to be determined by the pavement?s asphalt content (Tayebali and Huang, 2004). 3.5 Gradation GDOT uses GDT 38, Method of Test for Mechanical Analysis of Extracted Aggregate, as its method for determining the gradation of the aggregate samples. The sample of aggregate is sifted through a set of vertical sieves to determine the percent passing the specified sieves indicated earlier (GDOT, 2005). While it is easy to single out a property like asphalt content and link it to performance, it is more difficult to single out gradation. However, gradation is highly linked to the percentage of air voids in a mixture. The finer the gradation, the lower the percent air voids will be. The North Carolina State University study discussed earlier conducted an experiment where an SP 12.5 mm mix and an SP 19 mm mix were tested 29 for fatigue life. A typical mix and a mix with a 2% increase in air voids were the two samples in the experiment. A 2% increase in air voids caused the fatigue life of the SP 12.5 mm mix to have a 40% reduction, and the increase in air voids caused the SP 19 mm mix to have a 60% reduction in fatigue life. Therefore, the gradation of a mix can make a difference in the fatigue life of a pavement since it is tied to the air voids in the pavement (Tayebali and Huang, 2004). 3.6 Summary of Findings As shown in both the Kentucky and Auburn surveys, there are a variety of ways for states to organize their QA programs. Material properties, testing methods, and analysis methods are just three of the differences that can be seen from state to state. Terminology is another thing that varies from state to state. While the FHWA and TRB set up standard definitions, GDOT?s terminology varies somewhat. The relationships are show in Figure 3-1. Figure 3-1. Terminology Relationships Between Georgia DOT and TRB. Georgia specifically uses gradation and asphalt content as their properties tested by both the contractors and GDOT for acceptance. Every LOT requires one QCT test by GDOT QCT QA Comparison TRB Quality Control Acceptance Independent Assurance 30 the contractor, and two out of every five LOTs are tested by the DOT as an independent sample as QA. One out of every ten LOTs has a DOT comparison test from a split sample. This is graphically shown in Figure 3-2. Figure 3.2. Georgia DOT Sampling Ratios. LOTS 1 2 3 4 5 6 7 8 9 10 QCT 1 2 3 4 5 6 7 8 9 10 QA 1 2 3 4 (Independent Samples) Comparison 1 (Split Samples) 31 CHAPTER FOUR METHODOLOGY Two comparisons were made of contractor and GDOT test results. The first consisted of comparisons of the contractor?s QCT and GDOT?s QA test results. The second of comparisons of the contractor?s QCT results and GDOT?s comparison test results. The properties compared were asphalt content and percent passing the 1?, 0.75?, 0.5?, 0.375?, #4, #8, #50, and #200 sieves. 4.1 Data Management When the Georgia DOT sent the data to Auburn University?s Highway Research Center to be analyzed, the numbers needed for the analyses were contained in three Microsoft Access databases. Once it was understood which process each file contained, the organization of the column headings was not difficult to interpret. The difficulties arose in retrieving correct data in the project by project analyses. The overall analyses were easy as the analyzer could query a specific year and material property using built in programs in Microsoft Access. When the project by project analysis was conducted, each data row was visually inspected. A sort might have been available to speed up this process; however, many of the project titles were not consistent. The same project might have a ?0? (zero) instead of an ?O.? Another project 32 might have been listed by the contractor as ?0100,? but GDOT listed it as ?100.? This became especially difficult in trying to match projects from contractor to GDOT data. This inconsistency might be due to different employees recording the data in different ways; however, no conclusions can be made as to why these discrepancies occurred, but in order to keep accurate records in the GDOT data system, identical project number recording is imperative. A drop down menu providing a list of the projects in service at the time might be an appropriate way to help alleviate this problem. 4.2 Overall Data Analysis The first analysis was an overall data analysis. All of the 2003 data were divided among the various properties listed above, and then these results were put through a series of filters to find usable data. The first filter was one to exclude all of the data that did not include job mix formulas. This was accomplished through the use of queries in Microsoft Access. Only the data with job mix formulas could be used because target values were variable for each property; therefore, in order to combine data from job mix formulas, the target values had to be subtracted from the test results. The second filter applied to the data was outlier removal. Only obvious outliers, those impossible due to the next sieve size, could be removed from the data sets because there might have been inadequate mixes in the data set, and those mixes needed to be considered in the data analysis. One method for finding outliers was doing a sort of the data in Microsoft Excel. For example, if the results or job mix formula said 100, and the test results said 10 or 1 or vice versa, then the data was considered invalid, and it was removed. Once the outliers were removed, actual analyses began. 33 4.2.1 The F-Test The F-test is a test that analyzes the variances of the data sets in question. The F- test ?assesses mean differences by comparing the amounts of variability explained by different sources? (Ramsey and Shafer, 2002); therefore, the results of the test explain if there are differences in the variabilities of the data sets. The F-tests were run for each material property in both the QCT versus QA and QCT Comparison versus DOT analyses using all the data passing the two removal filters using the data analysis programs in Microsoft Excel. The null hypothesis was that the variances were not statistically different. Two values are typically used for confidence levels depending on the preference of the individual performing the analyses and how accurate the tests need to be. Those values are 95% and 99%. In the case of the F tests, and all of the other statistical tests for this analysis, a significance level of ?=0.01 was used; therefore, if the p-value was less than 0.01, the variances in the two data sets were considered different. This significance level was chosen to make finding differences between the sets more rigorous because fewer test results should be shown as different in a 1% significance level rather than a 5%. The results of the F-tests could be important for possibly two reasons. The results of the analysis would determine which type of t-test should be used for the remainder of the analyses for that specific material data set. Secondly, the results could possibly provide insight into altering of data. Despite the differences in the number of observations, the variances should still be similar; however, if one is significantly smaller than the other, it makes one wonder why? 34 4.2.2 The Three T-Tests Different forms of t-tests provide a way to determine whether the means of two data sets are statistically equal. However, in order for a t-test to be valid, at least three basic assumptions should be met. 1. The distribution of the data must be normal. 2. The two data sets must have equal standard deviations. 3. The two data sets must be independent. If these requirements are not met, a variation of the basic t-test is used (Ramsey et al., 2002). Each of these assumptions is dealt with in their own distinct way. When looking at normality, the Central Limit Theorem is used to justify the t-test in the case of looking at all of the data for the year. The Central Limit Theorem states, ?averages based on large samples have approximately normal sampling distributions, regardless of the shape of the population distribution? (Ramsey et al. 2002). Since the sample sizes were very large, it could be assumed that the distributions were normal. The second assumption states that each data set should have approximately the same standard deviation. This was determined using the F-test previously described. If the F-test returned a p-value less than 0.01, then the hypothesis that the two variances were equal was void, and a t-test based on equal variances in the data sets compared was not applicable. However, in this case, a t-test assuming unequal variances was used. There were some cases when testing the contractor QCT versus GDOT QA that this test was applied due to the F-test results. 35 The third assumption is that the data sets are independent of each other. While this was the case for the Contractor QCT versus GDOT QA test results, the QCT Comparison and GDOT Verification tests were conducted on split samples; therefore, this assumption was not met. In this case, a paired t-test was used. This test was a powerful application when it could be used, and it allowed plots to be created visually showing if the Contractor or GDOT had larger variances or mean discrepancies from the job mix formula. Lines of absolute numerical equality were drawn on the plots to make it easier to tell which values were larger. If all three assumptions were met, then a t-test was used for the contractor QCT versus GDOT QA comparison. Figure 4-1 provides a basic decision diagram to determine which test was appropriate for each analysis. 36 Figure 4-1. Analysis Thought Diagram. 4.2.3 Mean Squared Deviations While the F and t-tests provide individual analyses of the variances and means, respectively, calculating a mean squared deviation (MSD) for a data set quantifies both the accuracy and precision. Three possible scenarios can be used in calculating the mean square deviation of a data set: larger is better, nominal is better, and smaller is better. Nominal is better was chosen because observations can extend from either side of the target. The formula for the nominal is better scenario is expressed as Equation 1. The Data Sets Normal OK? NO YES Nonparametric Analysis Standard Deviation OK? Unequal Variances t-test NO YES Dependency OK? NO Paired t-test YES Basic t-test 37 nominal is better approach is the case where the results can approach from either the upper or the lower limit. 22 )( tnNIB xxSMSD ?+= (1) where: S n 2 = variance of data set _ x = mean of data set x t = target value for data set, 0 Smaller MSD values indicate a high level of control in the during the mix and construction process. In this study, MSD calculations were completed for all nine material properties for the contractor QCT, GDOT QA, contractor comparison, and GDOT comparison data. The smaller values of the four show which measurements were not only more precise but also closer to the target job mix formula. 4.2.4 Skewness The final analysis conducted on the overall project data was a skewness test. Skewness is a way to measure the symmetry of a data set when it has been converted into a histogram. If the skewness coefficient was zero, it did not necessarily mean that the data set was symmetrical; however, non-zero values do indicate how the data set?s histogram behaved. If the value of the coefficient is negative, then the data set?s histogram is skewed to the left. The converse is true for a positive skewness coefficient. Minitab was used to calculate skewness values. These tests were conducted to analyze possible data shifts. The Contractor?s QCT values were compared numerically to 38 those of GDOT?s QA and Comparison tests, but no statistical analyses, such as the standard error of skewness, were conducted on the actual skewness values. 4.3 Project by Project Analysis It was theorized that the results found for the overall data analyses might have been affected significantly by the large sample size; therefore, it was decided to conduct a further analysis for each of the large projects in Georgia?s database. A project was classified as large if the smallest project data set (contractor or DOT on a particular project) that contained had at least six usable entries. In the Contractor QCT versus GDOT QA analyses, the GDOT QA data set had to have six entries for that specific project, and for the QCT Comparison versus GDOT Verification analyses, six results just had to be recorded since split samples were used for this analysis. The decision to define a large project as a project with at least six test results was arbitrarily chosen. Three different properties were chosen for individual project analysis: asphalt content, the ?? sieve, and the #200 sieve. These two sieves were chosen because they were at the opposite ends of the gradation curve. F-tests were then run on all of the projects that had over six data entries to determine whether the variances of the data set were statistically the same at a 99% confidence level. Once the state of variances was determined, the appropriate t-test was applied to the Contactor QCT versus GDOT QA data set. Paired t-tests were run on the QCT Comparison versus GDOT Verification data. The difference in the data sets used in the individual versus overall project analysis was size; therefore, the Central Limit Theorem could not be applied to use as a justification 39 for the normality assumption required for the appropriate use of a t-test. However, it has been widely accepted that construction material property data tends to follow a normal distribution. In TRB?s Synthesis of Highway Practice 232: Variability in Highway Pavement Construction, completed in 1996, it was assumed that construction material data followed a normal curve. No other possible distribution was even considered (Hughes, 1996). Between 1996 and 1998, the Arkansas State Highway and Transportation Department and the University of Arkansas looked at the mix properties in asphalt construction in the state of Arkansas. One analysis included normality probability plots to check the normality of the construction data. The conclusion of this analysis was that ?the data generated by the sampling and testing program executed for this project were found to represent a population of results that follow a normal distribution? (Hall et al., 2002). Since it is generally accepted that construction material properties follows the normal distribution, the t-test, or a variation of such a test, was a valid analysis tool for the data sets in question. When the typical test results had the job mix formula subtracted from it, the means and variances were calculated during the F and t-test process using Microsoft Excel. Compiling these values on a project by project basis allowed for the creation of graphical comparisons for a specific material property for means and variances similar to those done in the overall project analysis with the paired t-tests. Lines of absolute numerical equality were used to show if the contractor?s data or GDOT?s data were farther from the target in the graphs and which sets of measurements had the largest variability. 40 The project by project analysis was conducted to see if there might be just a few projects influencing the results of the larger databases. However, if the majority of the projects showed significant differences, there might be deficiencies in the QA process in the state of Georgia. The project by project analysis also allowed for analyses in dealing with the tendencies of contractors to either have their results higher or lower than those of the DOTs in both means and variances. 4.4 Reduced Data Sets The reduced data sets were similar in appearance to the overall project analysis; however, the data set consisted of a smaller sample size. In this case, if an individual project was large enough to have undergone the individual analyses previously specified, its data were recompiled into a new database called the reduced data set. These data points were analyzed by F-tests and the appropriate t-test. This analysis was done to see if the larger sample size for the overall project analysis might have influenced the statistical tests used on the data sets. If different results were found, it could possibly mean that a few smaller projects were pulling the overall project data away from where it should have been. If the statistical differences noted for the different material properties were the same with the smaller sample size, the research team would feel more confident about proclaiming a statistical difference despite the possibility of sample size influence. 41 CHAPTER FIVE RESULTS As described in the methodology chapter, three different types of analyses were performed on the data provided by GDOT: overall project analyses, project by project analyses, and reduced data sets analyses. The overall data analyses consisted of all the data collected during the 2003 construction season for specific material properties. The project by project analyses included of all project that had at least 6 test results. These projects were individually subjected to various statistical tests. All of the data from individual projects that were analyzed were combined to make made up the reduced data set for analysis. 5.1 Overall Data Analysis Tables 5-1 and 5-2 provide summaries of the F and appropriate t-test results for comparing the Contractor QCT and GDOT QA data sets. The tables provide numerical values for sample sizes, variances, means of differences from job mix formulas, and p- values. The tables also include if statistical differences at the 99% level were found between the data sets and if the property is included in pay factor calculation. 42 Table 5-1. F-Test Results for Contractor QCT versus GDOT QA. Property n GDOT S 2 GDOT, % n CONT S 2 CONT, % Difference p-value Pay 1? Sieve 832 1.425 4775 1.296 No 0.034 No ?? Sieve 1637 4.167 9444 4.378 No 0.099 No ?? Sieve 2323 6.793 13157 5.565 Yes <0.001 Yes 3/8? Sieve 2099 6.605 11587 6.044 Yes 0.004 Yes #4 Sieve 1050 9.959 5532 7.707 Yes <0.001 Yes #8 Sieve 2488 9.488 14051 5.534 Yes <0.001 Yes #50 Sieve 749 4.139 4047 3.334 Yes <0.001 No #200 Sieve 2488 1.212 14036 0.769 Yes <0.001 No % Asphalt 2487 0.064 14061 0.040 Yes <0.001 Yes Table 5-2. t-Test Results for Contractor QCT versus GDOT QA. Property n GDOT ,% _ GDOT? n CONT ,% _ CONT? Difference p-value Pay 1? Sieve 832 0.187 4775 0.184 No 0.941 No ?? Sieve 1637 0.418 9444 0.535 No 0.036 No ?? Sieve 2323 0.196 13157 0.160 No 0.530 Yes 3/8? Sieve 2099 0.246 11587 0.231 No 0.805 Yes #4 Sieve 1050 0.320 5532 0.293 No 0.792 Yes #8 Sieve 2488 0.253 14051 0.196 No 0.380 Yes #50 Sieve 749 0.727 4047 0.837 No 0.170 No #200 Sieve 2488 0.359 14036 0.400 No 0.082 No % Asphalt 2487 0.004 14061 0.005 No 0.827 Yes As can be seen from Table 5-1, seven of the nine properties showed statistically significant differences in the variances while none of the means were shown to be statistically different in Table 5-2... The statistical differences in the variances might stem from the sizable sample sizes used in the analyses. The greater the sample size is, the more discriminating the test becomes; therefore, having sample sizes near or about 1000 might have caused the differences in the F-tests to be so statistically profound. While significant differences were shown in the statistical analyses, a simple visual inspection of the variances and means also was helpful in analyzing the data. Variances in the contractor?s data set were smaller than the variances of the GDOT data 43 set for eight of the nine material properties. This might lead some to believe the contractors are adjusting their results, even though slightly, closer to the target value especially since the only property where the DOT?s variance was smaller was not used in pay computations. However, while this difference shows up in the variances, only five of the nine contractor means are smaller for this data set, and none of the means from the two data sets show statistically significant differences for their means. Paired t-tests were run on the data sets comprised of split samples, QCT versus GDOT Comparison. Tables 5-3 and 5-4 provide the results of comparisons of variance and means for these data sets. Table 5-3. F-Test Results for Contractor QCT versus GDOT Comparison. Property N S 2 GDOT, % S 2 CONT, % Difference p-value Pay 1? Sieve 395 1.527 1.363 No 0.131 No ?? Sieve 791 4.410 3.831 No 0.024 No ?? Sieve 1067 9.343 6.576 Yes <0.001 Yes 3/8? Sieve 953 8.479 5.545 Yes <0.001 Yes #4 Sieve 402 9.450 8.606 No 0.175 Yes #8 Sieve 1142 8.673 6.561 Yes <0.001 Yes #50 Sieve 282 3.971 4.004 No 0.472 No #200 Sieve 1141 1.137 0.791 Yes <0.001 No % Asphalt 1135 0.088 0.045 Yes <0.001 Yes Table 5-4. t-Test Results for Contractor QCT versus GDOT Comparison. Property N ,% _ GDOT? ,% _ CONT? Difference p-value Pay 1? Sieve 395 0.258 0.295 No 0.462 No ?? Sieve 791 0.398 0.469 No 0.166 No ?? Sieve 1067 0.314 0.118 Yes 0.002 Yes 3/8? Sieve 953 0.516 0.329 Yes 0.005 Yes #4 Sieve 402 0.506 0.392 No 0.128 Yes #8 Sieve 1142 0.449 0.244 Yes <0.001 Yes #50 Sieve 282 0.897 0.763 No 0.094 No #200 Sieve 1141 0.334 0.447 Yes <0.001 No % Asphalt 1135 0.005 0.002 No 0.634 Yes 44 One would expect fewer differences between split sample results than between independent sample results since the same material is being tested. This seems to be the case with the variances where only five of the nine material properties show significant differences. Four of these properties are used by GDOT for payment decisions. However, while none of the means showed statistical differences in the independent samples, four of the nine p-values for the split sample means fell below the required 0.01 to be considered statistically equal. Once again, the variances from the contractor?s test results were consistently smaller than GDOT variances. The only property where the variance of the contractor was smaller was the #50 sieve, and that sieve is not included in the pay factor formula. In looking at the means, the contractor?s data were closer to the target value for only five of the nine properties; however, those five properties are the only five properties used in the pay calculations. This might once again lead one to suspect a possibility of data manipulation on the contractor?s part. To further investigate the split samples results, graphic representations were produced as shown in Figure 5-1 for asphalt content as an example. All of the graphs produced for the overall project analysis can be found in Appendix A. 45 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 x=DOT Comparison-JMF (%) Y= Co n t r a c t o r Q C T- JMF ( % ) Linear (Lines of Absolute Numerical Equality) Figure 5-1. Mean Comparison from Split Sample Results for Asphalt Content. The graphs are divided into four quadrants by two lines of absolute numerical equality to show which group?s test result was the largest. Each point in Figure 5-1 represents the GDOT Comparison test value and the contractor?s QC test value for a split sample. If the data point is contained in either of the quadrants also containing the horizontal axis, then the deviation from the target value for GDOT?s split sample results was larger than the contractor?s. If the data point is contained in either of the quadrants which also contains the vertical axis, then the converse is true. While many of the data points are located near the center of the graph, it can be seen that there are more points near the ends of the horizontal quadrants in comparison to the vertical quadrants. This supports the statistical evidence that the contractor?s test results for asphalt content were smaller in terms of both mean and variance. 46 Along with the F and t-tests, Means Squared Deviations (MSD) were calculated for all four data sets using the methodology described in Chapter 4. Table 5-5 shows the results from the calculations. The Contractor QCT and GDOT QA columns are the MSD results taken from independent samples while the Contractor QCT Comparison and GDOT comparison columns were calculated using the split sample test results. Table 5-5. MSD Results. Property Contractor QCT GDOT QA Contractor QCT Comparison GDOT Comparison 1? Sieve 1.330 1.460 1.450 1.594 ?? Sieve 4.664 4.342 4.051 4.568 ?? Sieve 5.591 6.831 6.590 9.442 3/8? Sieve 6.097 6.666 5.653 8.745 #4 Sieve 7.793 10.061 8.760 9.706 #8 Sieve 5.572 9.552 6.621 8.875 #50 Sieve 4.035 4.668 4.586 4.776 #200 Sieve 0.929 1.341 0.991 1.249 % Asphalt 0.040 0.064 0.045 0.088 The MSD test is used to compare both the variances and the means of the data sets in one term. As can be seen, either the contractor?s comparison or QCT test results have the smallest MSD value for every property. For eight of the nine properties, both of the contractor?s test results are smaller than either of GDOT?s values. One surprising trend in the data is seen when looking at the split samples MSD values. For the values at the ?? sieve, 3/8? sieve, #8 sieve, and asphalt content, there seems to be a considerable difference in the contractor?s and DOT?s values. However, these are also parameters where the variances were significantly different with a p-value that was less than 0.001. This supported a hypothesis that for these data sets the variance would probably be the dominant parameter in the equation. Many of the variances were values between 1 and 10, and all of the means were less than one. The means were then 47 squared in the equation making their significance even smaller; therefore, the correlation to the variance data is expected. When using the skewness programs in Minitab, the program produces both a skewness coefficient and the histogram of the dataset. Table 5-6 shows the skewness coefficients, and Figure 5-2 is an example of the histograms provided. The entire set of histograms is in Appendix B. Table 5-6. Skewness Results. Property Contractor QCT GDOT QA QCT Comparison GDOT Comparison 1? Sieve -0.377 -1.313 0.701 0.233 ?? Sieve 0.918 0.849 1.667 0.206 ?? Sieve 0.513 1.213 0.305 0.959 3/8? Sieve -0.351 -0.266 0.247 0.804 #4 Sieve 0.123 0.636 -0.204 -0.191 #8 Sieve 0.129 0.424 0.849 0.885 #50 Sieve 0.153 -0.480 -0.605 0.359 #200 Sieve -0.142 1.879 -0.385 -0.156 % Asphalt 0.105 0.969 0.444 0.593 1.300.65-0.00-0.65-1.30-1.95 Skewness 0.10473 N 14061 Mean 0.00534 StDev 0.20097 Figure 5-2. Skewness QCT-JMF for Asphalt Content. When comparing the independent samples, seven of the nine skewness values showed the absolute value of the GDOT QA skewness coefficient to be larger than that of the QCT; however, for the split sample test results, only four of the nine results were 48 larger for the contractor test results. It is difficult to assess the magnitude of these results as no standard errors of skewness were calculated. The most significant difference was found in the #200 sieve between the contractor QCT and the GDOT QA. The contractor?s data showed a slightly negative skew; however, GDOT?s skewness coefficient was highly positive. While these two sets of data did show significance differences in variances with the F-test analysis, the results would not necessarily lead one to expect the skewness results returned. 5.2 Project by Project Analysis 5.2.1 Statistical Analysis A project by project analysis was conducted for large projects on three material properties: ?? sieve, #200 sieve, and asphalt content. In order for a project to be considered for this analysis, it had to have at least 6 records for both of the comparative tests. When the data were sorted into projects, 114 projects were analyzed for the ?? sieves and asphalt content, and 126 projects were analyzed for the #200 sieve. Tables 5-7 through 5-10 provide summaries of the project-by-project results. The column titled ?Projects with Significantly Higher GDOT Variances? contains two percentages in it. The first percentage is a comparison of the projects with significantly higher GDOT variances with all the projects while the second number is a comparison to only the projects with significant differences. The detailed summaries of the project-by-project analyses can be found in Appendix C. 49 Table 5-7. Contractor QCT versus GDOT QA Variance Results by Project. Property Projects Projects with Larger GDOT Variances Projects with Significant Differences Projects with Significantly Higher GDOT Variances % Asphalt 114 77 (68%) 12 (10%) 10 (9%) (83%) ?? Sieve 114 63 (55%) 13 (11%) 10 (9%) (77%) #200 Sieve 126 81 (64%) 17 (13%) 15 (12%) (88%) Table 5-8. Contractor QCT versus GDOT QA Mean Results by Project. Property Projects Projects with Larger GDOT Means Projects with Significant Differences Projects with Significantly Higher GDOT Means % Asphalt 114 68 (60%) 8 (7%) 6 (5%) (75%) ?? Sieve 114 50 (54%) 3 (3%) 3 (3%) (100%) #200 Sieve 126 52 (41%) 13 (10%) 6 (5%) (46%) Table 5-9. Contractor Comparison verses GDOT Comparison Variance Results by Project. Property Projects Projects with Larger GDOT Variances Projects with Significant Differences Projects with Significantly Higher GDOT Variances % Asphalt 41 35 (85%) 1 (2%) 1 (2%) (100%) ?? Sieve 34 20 (59%) 2 (6%) 2 (6%) (100%) #200 Sieve 45 34 (76%) 3 (7%) 3 (7%) (100%) Table 5-10. Contractor Comparison verses GDOT Comparison Mean Results by Project. Property Projects Projects with Larger GDOT Means Projects with Significant Differences Projects with Significantly Higher GDOT Means % Asphalt 41 27 (66%) 1 (2%) 0 (0%) (0%) ?? Sieve 34 15 (44%) 0 (0%) 0 (0%) (0%) #200 Sieve 45 21 (47%) 2 (4%) 2 (4%) (100%) When looking at the results from the Contractor QCT and GDOT QA project by project analysis, it can be seen that projects where GDOT had larger means ranged from 41% to 60%. On the other hand, GDOT had larger variances for between 55% and 68% of the projects. Twenty-four of the 354 projects had significant differences in means, and 50 for 15 (63%) of these, GDOT QA had the larger mean. For variances, 42 of the 354 projects had statistical differences, and for 35 (83%) of those, GDOT QA variances were the largest. When looking at Tables 5-9 and 5-10, one can see that the results of the split sample comparisons on a project by project basis were similar to those found for the independent samples. Asphalt content sieve was the only property showing a greater significant deviation from the target by GDOT. GDOT values for all three properties were larger from 44% to 66%. However, GDOT variances were larger between 59% and 85% of the projects. Only 3 of the 120 projects had significant differences between mean values, and two of those showed larger GDOT means. Six of the 120 projects had significant differences in the variances, and all six of those projects had larger GDOT variances. Scatterplots, as shown in Figures 5-3 and 5-4, were created to analyze the created data sets. The results of the GDOT QA would be graphically represented against the Contractor?s QCT to show mean deviations and variances. One of the useful results from a graphical approach would be the easy recognition of outliers that might appear in the data set where one agency?s test results were vastly different from those of the other. All the created scatterplots can be found in Appendix D. 51 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 QA QC T Figure 5-3. Asphalt Content Variances for GDOT QA versus Contractor QCT. -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 QA QC Figure 5-4. Asphalt Content Mean Deviations for GDOT QA versus Contractor QCT. 52 5.2.2 Precision Tests One of the places were variances can occur is in the test itself. The American Association of State Highway and Transportation Officials (AASHTO) conducted a series of experiments on the testing methods themselves to determine appropriate levels of variation that might be explained by test itself. For the measuring of asphalt content, Georgia allows its contractors to test the samples by either the ignition oven or the extraction method. According to AASHTO, the allowable standard deviation for a multi-laboratory ignition test is 0.06%, and the allowable standard deviation for a multi-laboratory extraction tests is 0.29%. In order to determine if the project?s test fell outside that range, a pooled standard deviation was calculated. Since Georgia?s data set did not specify which test was used, the extreme case was used. It was assumed that both methods were used for every project. Table 5- 11 provides the results of the test (AASHTO, 2004). Table 5-11. Asphalt Content Precision Test Method Independent Sample Projects Number of Projects Outside Test Precision Split Sample Projects Number of Projects Outside Test Precision Ignition 114 114 (100%) 41 41 (100%) Extraction 114 7 (6%) 41 10 (24%) Every project tested, whether from split or independent sample, fell outside the test precision for the ignition method; however, fewer fell outside the precision for the extraction method. The precision for gradation was not dependent upon sieve size. It was dependent upon the percent passing the sieve. An average of the percent passing the sieve in 53 question was calculated to determine the allowable standard deviation for the test. As with asphalt content, a pooled standard deviation was calculated using both the contractor and the GDOT data to compare to the allowable precision of the test (AASHTO, 2004). A summary of the results is in Table 5-12. The results of this analysis once again show the test standard deviations consistently falling outside the allowable precision of the test for a multi-laboratory test. Table 5-12. Gradation Precision Property Independent Sample Projects Number of Projects Outside Test Precision Split Sample Projects Number of Projects Outside Test Precision ?? Sieve 114 101 (89%) 34 31 (91%) #200 Sieve 126 96 (76%) 45 36 (80%) 5.3 Reduced Data Set Analysis The third analysis was done by combining data from individual projects that were used for the project by project analysis into appropriate contractor or GDOT data sets. The data sets were compared with F and t-tests, and Tables 5-13 and 5-14 provide the results. Table 5-13. Reduced Data Set Analysis for GDOT QA versus Contractor QCT. Property Projects n G S 2 G ? G n C S 2 C ? C Difference % Asphalt 114 1410 0.058 0.011 8453 0.040 0.010 Variances ?? Sieve 114 1385 7.701 0.146 8072 6.439 0.208 Variances #200 Sieve 126 1565 1.210 0.310 8908 0.741 0.367 Variances When considering the QA versus QCT data, one can see that none of the means were significantly different, but all three variances were seen to be significantly different. These results parallel those found in the overall project analysis. In the case of variances, the GDOT QA results were always larger than those for the contractor. 54 Table 5-14. Reduced Data Set Analysis for GDOT Comparison versus Contractor QCT Comparison. Property Projects N S 2 GDOT ? GDOT S 2 CONT ? CONT Differences % Asphalt 41 452 0.097 0.018 0.053 0.010 Variances ?? Sieve 35 400 12.286 0.462 9.251 0.200 Variances #200 Sieve 45 470 7.870 0.159 7.613 0.278 Means The results of the split sample data sets were different that those from independent samples. The contractor?s variances are always smaller than those of GDOT, but only variances for percent asphalt and percent passing the ?? sieve were significantly different. The #200 sieve had variances that were statistically similar; however, the means were significantly different with the contactor having the larger mean deviation. 5.4 Summary of Findings Statistically, the contractors and GDOT have similar means, but as seen in previous studies, the variances are where differences occur. In all three analyses, the variances were more likely to be seen as statistically significant than the means were. On the project level, it was seen that the majority of the projects had the contractor?s variances being smaller than those of GDOT. A precision analysis was also conducted on project by project level to determine if the results were falling within the allowable standard deviations of the test. The analysis concluded the only test having a possibility of fewer than 76% of the projects falling within test precision was the test to determine asphalt content by extraction; however, the specification test methods are not known. 55 56 CHAPTER SIX CONCLUSIONS In doing an analysis such as the one described in this report, it is possible that some of the obtained results were not what was expected. On the other hand, other analyses might have played directly into preconceived ideas about what the results might be. The following paragraphs will provide conclusions and recommendations for data organization, overall and reduced project analyses, and project by project analyses. 6.1 Overall Data and Reduced Data Set Analyses The overall data analyses were conducted on all data that had a corresponding job-mix formula for the year 2003. The reduced data set consisted of data from all projects that had at least six or more QA and/or Comparison tests results. Tables 6-1 and 6-2 provide insights into the extent of the analysis. These data sets were compared using F and t-tests as described in Chapter 4. Table 6-3 provides a brief summary of the results from these analyses indicating when differences in means or variances were statistically significant at the 99% level. Detailed results can be found in Chapter 5. 57 Table 6-1. Sample Sizes for Overall Data Results. Independent Samples Split Samples Property N GDOT N CONT N 1? Sieve 832 4775 395 ?? Sieve 1637 9444 791 ?? Sieve 2323 13157 1067 3/8? Sieve 2099 11587 953 #4 Sieve 1050 5532 402 #8 Sieve 2488 14051 1142 #50 Sieve 749 4047 282 #200 Sieve 2488 14036 1141 % Asphalt 2487 14061 1135 Table 6-2. Sample Sizes for Reduced Data Set Results Independent Samples Split Samples Property N GDOT N CONT N ?? Sieve 1385 8072 400 #200 Sieve 1565 8908 470 % Asphalt 1410 8453 452 Table 6-3. Summary of Results for Overall Data Analyses and Reduced Data Set Analyses. Overall Data Analysis Reduced Data Set Analysis Property QCT vs. QA QCT vs. DOT Comparison QCT vs. QA QCT vs. DOT Comparison 1? Sieve No Differences No Differences ?? Sieve No Differences No Differences ?? Sieve Variances Variances and Means Variances Variances 3/8? Sieve Variances Variances and Means #4 Sieve Variances No Differences #8 Sieve Variances Variances and Means #50 Sieve Variances No Differences #200 Sieve Variances Variances and Means Variances Means Asphalt Content Variances Variances Variances Variances 58 As can be seen from the table, when variances were found to be statistically different in one analysis, the other analysis seemed to follow suit. The elimination of small projects from the overall data set to the reduced data set had little impact on the statistical significance of differences. The results were what were expected based upon the Kentucky and Alabama studies described in Chapter 2. One theory for the majority of the material properties exhibiting statistical differences at the 99% confidence interval is the size of the samples used in the study. The higher the number of degrees of freedom, the more confining the test is going to become. Another theory could also possibly explain the statistical differences seen above. If a few of the larger projects had significant statistical differences, the results of the overall analysis could be swung in a favorable or unfavorable direction towards statistical differences. The split sample results show at least one statistically significant difference, whether for mean or variance, for 4 of the 5 pay properties. For these five properties, whether a statistical difference was noted or not, the contractor?s mean value was always closer to the target, and its variance was always smaller than that provided by the DOT. The results of the MSD and skewness analyses seemed to reiterate the findings of the F and t-tests. The contractors were consistently more accurate and precise in their test results. 6.2 Project by Project Analysis In order for a project to be considered in the project analysis, it had to have at least six QA test results and/or Comparison tests. These analyses were conducted on the 59 ?? sieve, #200 sieve, and asphalt content. The detailed results are found in Chapter 5. The most thought provoking data coming from these analyses were the project variance results. While overall only a small percentage of the projects showed a statistical difference in the two data sets for variances, GDOT had the larger of the variances for the majority of the projects for all properties considered and all sample types. One would think the split sample results would have been closer since they were analyzing the testing methods and not material properties; however, the split sample results seen in Table 6-5 are even more lopsided that those for the independent samples. For the asphalt content projects, GDOT variances were larger for 85% of the projects. Another result to consider is how many of the projects with significant differences had larger GDOT variances. As said before, while there were only a few projects with statistical differences, over all three properties considered 83% of the projects containing statistical differences had larger GDOT variances for independent samples. The split samples had 100% of the 6 projects with larger GDOT variances if a statistical difference was noted. Tables 6-4 and 6-5 summarize the individual project analyses. Table 6-4. QCT versus QA Results Summary. Property Projects Projects with Larger GDOT Variances Projects with Significant Variance Differences Projects with Significantly Higher GDOT Variances Projects with Larger GDOT Means Projects with Significant Mean Differences Projects with Significantly Higher GDOT Means ?? Sieve 114 63 (58%) 13 (11%) 10 (9%) 50 (54%) 3 (3%) 3 (3%) #200 Sieve 126 81 (64%) 17 (13%) 15 (12%) 52 (41%) 13 (10%) 6 (5%) % Asphalt 114 77 (68%) 12(10%) 10 (9%) 68 (60%) 8 (7%) 6 (5%) 60 Table 6-5. Comparison Results Summary. Property Projects Projects with Larger GDOT Variances Projects with Significant Variance Differences Projects with Significantly Higher GDOT Variances Projects with Larger GDOT Means Projects with Significant Mean Differences Projects with Significantly Higher GDOT Means ?? Sieve 34 20 (59%) 2 (6%) 2 (6%) 15 (44%) 0 (0%) 0 (0%) #200 Sieve 45 34 (76%) 3 (7%) 3 (7%) 21 (47%) 2 (4%) 2 (4%) % Asphalt 41 35 (85%) 1 (2%) 1 (2%) 27 (66%) 1 (2%) 0 (0%) One would think these percentages for both the independent samples and the split samples would be closer to 50%, but for some reason, the contractor?s numbers seem to be consistently closer to the value specified in the JMF. This might be due to improved testing ability with frequent testing, or it might be a symptom of a larger problem. 6.3 Final Concerns and Recommendations ? A data management system should be set up to control the proper recording of project numbers. This could be accomplished by either having one person input all of the data for one project or by having a menu to select projects from. ? The properties selected by the state of Georgia to incorporate into their QA plan could possibly be amended. At the present time, contractor testing is only done on mix properties while GDOT tests for mat density and smoothness. Many states incorporate a contractor-performed mat density test which might be useful in making sure the product is laid and compacted properly. Another concern might arise in the number of sieves incorporated into the pay factor. If one of the upper sieves is collects too high or too low a percentage of the aggregates, then 61 the corresponding sieves below will also be off in a compounding nature. One mistake in an upper sieve might prove harmful to the actual pay of the project while not being very detrimental to the performance of the mix. Georgia has four consecutive sieves that are used for pay factor. This probably leads to a high correlation between the sieves. Spreading out the sieves used in the pay factor computation would decrease the correlation and might lead to a better representation of the mix. ? During the precision analysis, it was seen that the standard deviations for the projects were consistently falling outside the allowable standard deviation based upon precision tests conducted on the testing method. The extraction method was the only test that did not have 75% or more of its tests falling outside the allowable standard deviation. More tests failed to fall within precision limits than contained statistically significant differences. ? The biggest concern with the Georgia DOT data at this time is the variance results reported. At the present time, Georgia accepts data based upon a percentage value, but no statistical analysis is completed. Some states conduct F and t-tests in order to determine if the material is acceptable or not. It might be worthwhile for Georgia to use these statistical tests to keep a closer eye on its quality control/quality assurance process. If not these statistical analyses, then some other means of monitoring more than just the means should be employed. 62 REFERENCES AASHTO. Standard Specifications for Transportation Materials and Methods of Sampling and Testing. AASHTO: Washington, D.C., 2004. FHWA. 23 CFR Part 637. Washington D.C., 1995. GDOT. GSP 21. . October 21, 2005. Hall, Kevin D. and Stacy G. Williams. ?Establishing Variability for Hot-Mix Asphalt Construction in Arkansas.? Journal of the Transportation Research Board 1813: 2002. Hancher, Donn E., Yuhong Wang, and Kamyar C. Mahboub. Contractor Performed Quality Control on KyTC Projects. Kentucky Transportation Center: Lexington, Kentucky: 2002. Mahboub, Kamyar C., Donn E. Hancher, and Yuhong Wang. ?Contractor-Performed Quality Control: Is the Fox Guarding the Henhouse?? Journal of Professional Issues in Engineering Education and Practice Volume 130:4. ASCE: October 2004. Mahboub, Kamyar C. and Donn E. Hancher. Development of Concrete QC/QA Specifications for Highway Construction in Kentucky. Kentucky Transportation Center: Lexington, Kentucky, 2001. 63 Parker, Frazier, Jr. and Md. Shabbir Hossain. ?Hot-Mix Asphalt Mix Properties Measured for Construction Quality Control and Assurance.? Journal of the Transportation Research Board 1469: 1995. Parker, Frazier, Jr. and M. Shabbir Hossain. ?Statistics for Superpave Hot-Mix Asphalt Construction Quality Control/Quality Assurance Data.? Journal of the Transportation Research Board 1813: 2002. Ramsey, Fred L. and Daniel W. Schafer. The Statistical Sleuth. Pacific Grove, California: 2002. Schmitt, Robert L., Awad S. Hanna, Jeffrey S. Russel, and Erik V. Nordheim. ?Analysis of Bias in HMA Field Split-Sample Testing.? Journal of the Association of Asphalt Paving Technologists. Volume 70: 2001. Sholar, Gregory A., Gale C. Page, James A. Musselman, Patrick B. Upshaw, and Howard L. Moseley. ?Development of the Florida Department of Transportation?s Percent Within Limits Hot-Mix Asphalt Specification.? TRB 84 th Annual Meeting Compendium of Papers CD-ROM: 2005. Tayekabli, Akhtarhusein and Yuanxiong Huang. Material Characterization and Performance Properties of Superpave Mixtures. FHWA: Washington, D.C., 2004. Transportation Research Board. Glossary of Highway Quality Assurance Terms. Washington D.C., 2002. 64 APPENDICES 65 APPENDIX A OVERALL DATA ANALYSIS SCATTERPLOTS -8 -6 -4 -2 0 2 4 6 8 -8 -6 -4 -2 0 2 4 6 8 x = DOT-JMF (% Passing) y = Q C T - JM F ( % Pas s i n g) Linear (Line of Absolute Numerical Equality) Figure A-1. Mean Comparison from Split Sample Results for 1? Sieve. -20 -15 -10 -5 0 5 10 15 20 -20-15-10-5 5 101520 x = DOT-JMF (% Passing) y = Q C T- J M F ( % P a ssi n g ) Linear (Line of Absolute Numerical Equality) Figure A-2. Mean Comparison from Split Sample Results for 3/4? Sieve. 66 -25 -20 -15 -10 -5 0 5 10 15 20 25 -25 -20 -15 -10 -5 0 5 10 15 20 25 x = DOT-JMF (% Passing) y = Q C T - J M F (% Pas s i ng ) Linear (Line of Absolute Numerical Equality) Figure A-3. Mean Comparison from Split Sample Results for 1/2? Sieve. -15 -10 -5 0 5 10 15 -15 -10 -5 0 5 10 15 x = DOT-JMF (% Passing) y = QC T - JM F (% Pas s i ng ) Linear (Line of Absolute Numerical Equality) Figure A-4. Mean Comparison from Split Sample Results for 3/8? Sieve. 67 Figure A-5. Mean Comparison from Split Sample Results for #4 Sieve. -25 -20 -15 -10 -5 0 5 10 15 20 25 -25 -15 -5 5 15 25 x = DOT-JMF (% Passing) y = Q C T - J M F ( % Pa s s i n g ) Linear (Line of Absolute Numerical Equality) Figure A-6. Mean Comparison from Split Sample Results for #8 Sieve. -15 -10 -5 0 5 10 15 -15 -10 -5 0 5 10 15 x = DOT-JMF (% Passing) y = QCT - JM F ( % Pas s i ng) Linear (Line of Absolute Numerical Equality) 68 -10 -8 -6 -4 -2 0 2 4 6 8 10 -10 -5 0 5 10 x = DOT-JMF (% Passing) y = Q C T - J M F ( % Pa s s i n g ) Linear (Line of Absolute Numerical Equality) Figure A-7. Mean Comparison from Split Sample Results for #50 Sieve. -6 -4 -2 0 2 4 6 -6 -4 -2 0 2 4 6 x = DOT-JMF (% Passing) y = QC T - JM F (% Pa s s i ng) Linear (Line of Absolute Numerical Equality) Figure A-8. Mean Comparison from Split Sample Results for #200 Sieve. 69 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 x = DOT-JMF (Aspalt Content) y = QC T - J M F (As pha l t C o nte n t) Linear (Line of Absolute Numerical Equailty) Figure A-9. Mean Comparison from Split Sample Results for Asphalt Content. 70 APPENDIX B SKEWNESS ANALYSIS RESULTS 1.300.65-0.00-0.65-1.30-1.95 Skewness 0.10473 N 14061 Mean 0.00534 StDev 0.20097 Figure B-1. QCT-JMF Asphalt Content 1.300.65-0.00-0.65-1.30-1.95 Skewness 0.9689 N 2487 Mean 0.00416 StDev 0.25333 Figure B-2. QA-JMF Asphalt Content 1.81.20.60.0-0.6-1.2-1.8 Skewness 0.44370 N 1135 Mean 0.00211 StDev 0.21171 Figure B-3. QCT-JMF Asphalt Content (Split Sample) 71 1.81.20.60.0-0.6-1.2-1.8 Skewness 0.59261 N 1135 Mean 0.00547 StDev 0.29696 Figure B-4. DOT-JMF Asphalt Content (Split Sample) 6.44.83.21.60.0-1.6-3.2-4.8 Skewness -0.142382 N 14306 Mean 0.39994 StDev 0.87716 Figure B-5. QCT-JMF #200 Sieve 6.44.83.21.60.0-1.6-3.2-4.8 Skewness 1.8794 N 2488 Mean 0.3594 StDev 1.1011 Figure B-6. QA-JMF #200 Sieve 72 4.53.01.50.0-1.5-3.0-4.5-6.0 Skewness -0.38469 N 1141 Mean 0.44722 StDev 0.88951 Figure B-7. QCT-JMF #200 Sieve (Split Sample) 4.53.01.50.0-1.5-3.0-4.5-6.0 Skewness -0.15567 N 1141 Mean 0.33364 StDev 1.06621 Figure B-8. DOT-JMF #200 Sieve (Split Sample) 10.57.03.50.0-3.5-7.0-10.5-14.0 Skewness 0.152838 N 4047 Mean 0.8367 StDev 1.8258 Figure B-9. QCT-JMF #50 Sieve 73 10.57.03.50.0-3.5-7.0-10.5-14.0 Skewness -0.48027 N 749 Mean 0.7274 StDev 2.0343 Figure B-10. QA-JMF #50 Sieve 9630-3-6-9 Skewness -0.60465 N 282 Mean 0.7628 StDev 2.0010 Figure B-11. QCT-JMF #50 Sieve (Split Samples) 9630-3-6-9 Skewness 0.358621 N 282 Mean 0.89681 StDev 1.99264 Figure B-12. DOT-JMF #50 Sieve (Split Samples) 74 24181260-6-12 Skewness 0.12858 N 14051 Mean 0.1958 StDev 2.3525 Figure B-13. QCT-JMF #8 Sieve 24181260-6-12 Skewness 0.4244 N 2488 Mean 0.2528 StDev 3.0802 Figure B-14. QA-JMF #8 Sieve 20151050-5-10-15 Skewness 0.84939 N 1142 Mean 0.2439 StDev 2.5614 Figure B-15. QCT-JMF #8 Sieve (Split Samples) 75 20151050-5-10-15 Skewness 0.88454 N 1142 Mean 0.4494 StDev 2.9451 Figure B-16. DOT-JMF #8 Sieve (Split Samples) 22.016.511.05.50.0-5.5-11.0 Skewness 0.12328 N 5532 Mean 0.2930 StDev 2.7762 Figure B-17. QCT-JMF #4 Sieve 22.016.511.05.50.0-5.5-11.0 Skewness 0.63557 N 1050 Mean 0.3205 StDev 3.1558 Figure B-18. QA-JMF #4 Sieve 76 1284-0-4-8-12 Skewness -0.20406 N 402 Mean 0.3820 StDev 2.9335 Figure B-19. QCT-JMF #4 Sieve (Split Samples) 1284-0-4-8-12 Skewness -0.19096 N 402 Mean 0.5065 StDev 3.0740 Figure B-20. DOT-JMF #4 Sieve (Split Samples) 15105-0-5-10-15 Skewness -0.35097 N 11587 Mean 0.2312 StDev 2.4585 Figure B-21. QCT-JMF 0.375? Sieve 77 15105-0-5-10-15 Skewness -0.26617 N 2099 Mean 0.2461 StDev 2.5700 Figure B-22. QA-JMF 0.375? Sieve 20151050-5-10 Skewness 0.24666 N 953 Mean 0.3285 StDev 2.3548 Figure B-23. QCT-JMF 0.375? Sieve (Split Samples) 20151050-5-10 Skewness 0.80359 N 953 Mean 0.5162 StDev 2.9118 Figure B-24. DOT-JMF 0.375? Sieve (Split Samples) 78 28211470-7-14 Skewness 0.51317 N 13157 Mean 0.1596 StDeva 2.3591 Figure B-25. QCT-JMF 0.5? Sieve 28211470-7-14 Skewness 1.2129 N 2323 Mean 0.1960 StDeva 2.6063 Figure B-26. QA-JMF 0.5? Sieve 181260-6-12-18 Skewness 0.30511 N 1067 Mean 0.1176 StDeva 2.5644 Figure B-27. QCT-JMF 0.5? Sieve (Split Samples) 79 181260-6-12-18 Skewness 0.95880 N 1067 Mean 0.3137 StDev 3.0566 Figure B-28. DOT-JMF 0.5? Sieve (Split Samples) 10.57.03.5-0.0-3.5-7.0-10.5-14.0 Skewness 0.91821 N 9444 Mean 0.5348 StDev 2.0923 Figure B-29. QCT-JMF 0.75? Sieve 10.57.03.5-0.0-3.5-7.0-10.5-14.0 Skewness 0.84905 N 1637 Mean 0.4176 StDev 2.0413 Figure B-30. QA-JMF 0.75? Sieve 80 12840-4-8 Skewness 1.66709 N 791 Mean 0.4688 StDev 1.9574 Figure B-31. QCT-JMF 0.75? Sieve (Split Samples) 12840-4-8 Skewness 0.2056 N 791 Mean 0.3979 StDev 2.1000 Figure B-32. DOT-JMF 0.75? Sieve (Split Samples) 6420-2-4-6-8 Skewness -0.37664 N 4775 Mean 0.18421 StDev 1.13825 Figure B-33. QCT-JMF 1? Sieve 81 6420-2-4-6-8 Skewness -1.3132 N 832 Mean 0.1874 StDev 1.1937 Figure B-34. QA-JMF 1? Sieve 420-2-4-6 Skewness 0.70112 N 395 Mean 0.29468 StDev 1.16758 Figure B-35. QCT-JMF 1? Sieve (Split Samples) 420-2-4-6 Skewness 0.23251 N 395 Mean 0.25848 StDev 1.23553 Figure B-36. DOT-JMF 1? Sieve (Split Samples) 82 APPENDIX C PROJECT BY PROJECT TABLES 83 Table C-1. ?? Sieve Project by Project Table for QCT versus QA. Project # Nqc Nqa Vqc Vqa Mqc Mqa Differences BRN-42-2(40)01 13 6 1.7856 1.2457 -1.1308 -1.1167 None CM-186-1(28)01 38 7 11.1322 13.2024 0.6421 0.9286 None CSSTP-M002- 00(444)01 27 6 0.2972 1.9417 2.5519 2.4167 Variances CSSTP-M002- 00(453)01 54 7 0.0007 0.0000 -0.0037 0.0000 None EDS-27(136)01 138 35 4.8818 28.2736 0.3449 1.6229 Variances EDS-27(147)01 90 14 2.3432 1.5782 -1.1922 -1.0857 None EDS-84(16)01 127 16 7.3232 5.3740 0.6583 0.5750 None EDS-84(17)01 64 11 6.4686 9.9626 0.6000 0.2636 None EDS-545(8)01 71 9 7.6223 4.2553 -0.4099 0.0556 None EDS-555(6)01 73 15 7.8346 12.1884 -0.0603 1.0533 None EDS-565(7)-01 99 7 7.8370 13.9381 0.4657 0.4143 None EDS-565(15)-01 98 15 5.7659 8.2995 0.8276 0.7333 None FLF-540(8)01 61 8 5.8679 4.1250 1.0672 1.2750 None FLF-540(25)01 95 12 3.6104 3.6924 1.4168 0.9167 None GIP-341(33)01 107 13 9.2501 9.6309 0.8533 0.6385 None GIP-341(34)01 146 25 14.9424 27.4157 0.3671 -0.7640 None GIP-341(38)01 203 21 14.0092 10.8481 0.9163 1.1714 None HPP-3717-00(300)01 28 16 0.0714 0.1873 -0.3786 -0.3937 None HPP-STP-178-1(33)01 23 7 2.1225 0.9657 0.4391 0.3714 None IM-00MS(268)01 41 14 4.1352 8.1385 -0.2317 -0.4000 None IM-285-1(349)01 26 7 13.5448 113.5148 -0.7346 1.3857 Variances IM-0000-00(471)01 54 15 10.2602 6.1898 1.0593 1.5467 None IM-0000-00(470)01 87 11 10.2481 13.3836 1.4839 1.3182 None IM-00MS(20)01 56 17 5.4579 8.8272 1.3750 1.4706 None IM-MS(325)01 31 7 5.9792 2.8800 -1.5484 -2.1000 None IM-NH-85-2(148)01 175 20 10.5233 11.2510 1.1863 1.0950 None LAR00-S005- 00(593)C1 13 8 0.4990 0.4536 0.2308 0.3250 None LAR00-S005- 00(649)C1 13 6 0.0000 0.0067 0.0000 -0.0333 Variances LAR31-87-1(067)01 23 7 0.0157 0.0057 -0.0261 -0.0286 None LAR32-2-3(067)01 17 7 0.3368 0.3262 -0.3059 -0.3571 None LAR32-2-5(63)C1 12 6 0.0000 0.0150 0.0000 -0.0500 Variances LAR32-24-1(057)C1 42 8 0.7603 0.4570 -0.3214 0.0625 None LAR32-30-2(89)01 34 9 29.7617 28.5753 0.9206 -2.5556 None LAR32-250-1(057)01 37 8 0.6825 0.5998 -0.5973 -0.2625 None LAR32-888-1(089)01 35 11 3.7732 7.0876 0.4971 1.7182 None LAU32-8532-69(121)01 28 10 1.0177 0.0573 -0.6929 -0.2200 Variances NH-8013(8)01 63 20 8.3353 4.9237 1.7175 0.4450 None NH-11-1(44)01 32 11 11.3519 2.8787 1.7188 -0.8545 None NH-012-1(83)01 60 10 11.4486 17.9107 0.6417 0.1800 None NH-017-2(53)01 36 12 10.8882 16.4717 -0.3583 -0.2417 None NH-20-2(183)01 193 24 7.0404 8.6573 0.5041 2.3083 Mean 84 NH-026-2(81)01 29 9 7.7946 8.5050 0.6034 -0.2333 None NH-043-1(49)01 14 6 9.5191 4.6387 -0.5286 -0.6667 None NH-56-1(62)01 43 16 15.4562 17.5320 1.6372 -0.6563 None NH-75(157)01 234 30 10.0084 7.7081 0.3910 -0.2767 None NH-75-1(203)01 180 17 9.4872 17.3468 1.1406 1.1059 None NH-171-1(4)01 100 8 4.4169 2.1943 2.1360 1.3000 None NH-IM-75-1(158) 174 24 9.0923 8.5748 -0.9506 -0.9208 None NH-IM-95-1(119)01 90 13 9.5126 23.0640 -0.2078 -1.7308 Variances NH-IM-95-1(125)01 141 29 4.7353 5.3664 -0.6099 -0.5000 None NH-IM-95-1(155)01 71 11 15.2213 9.6702 0.7423 3.5273 None NHS-0000-00(768)01 61 8 7.8002 11.5813 2.0426 3.4125 None NHS-M002-00(398)01 487 53 5.3242 5.2975 0.0031 -0.0434 None NHS-M001-00(530)01 192 9 13.0622 11.9128 1.5516 1.6556 None NHS-M001-00(531)01 59 12 1.8749 3.7355 -0.8305 -0.5500 None NHS-M001-00(591)01 50 13 3.5058 3.4500 -1.5560 -0.2000 None NHS-M002-00(276)01 23 8 2.0715 3.9200 1.2652 0.9000 None NHS-M002-00(292)01 247 10 5.9524 9.4250 -0.0142 -0.0180 None PR000-S005- 00(810)C1 18 7 0.3657 0.0129 0.1278 -0.0429 Variances PR-147-2(055)01 8 7 0.0000 0.0000 0.0000 0.0000 None PR131-1(63)C1 8 6 0.0000 0.0000 0.0000 0.0000 None PR-1963-2(139)01 28 6 7.1455 12.4910 1.7536 2.0500 None PR-8530-83(39)C1 22 10 1.1089 5.3044 0.9682 0.6000 Variances PR-8531-21(77)C1 51 11 12.8023 14.1076 0.9078 0.6818 None PRLOP8530-32(15)C1 31 6 5.4603 3.7147 0.0032 -0.4333 None SAMA-60SP(4)01 30 6 2.8886 5.6800 0.2800 1.2000 None SAMA-3(249)01 38 9 6.5743 3.4344 -0.1974 2.5222 Mean SAMA-3(294)01 123 25 1.1791 1.0867 -0.7772 -0.8400 None SAMA-3(298)01 198 31 2.3558 1.3203 0.3535 0.4677 None SAMA-39(44)01 55 14 4.4110 0.9455 -0.4709 -0.3643 Variances SAMA-53(133)01 34 6 0.6020 0.6787 0.3529 0.3333 None SAMA-74(56)01 25 9 1.2208 0.8700 1.5920 0.8333 None SAMA-74(58)01 121 23 1.2860 0.8421 -1.2281 -1.2130 None SAMA-155(52)01 18 6 0.7662 2.3680 -1.1167 -2.8000 Mean SAMA-206-C0(3)01 45 7 0.6860 1.0795 0.1244 -0.0429 None SAMA-234-(17)01 51 8 0.6792 0.9484 2.0608 1.8625 None SAMA-520(58)01 84 18 0.4159 0.5603 0.3798 0.4444 None SAMA-520(59)01 144 20 0.7411 0.7887 -0.3514 -0.3850 None SAM-M002-00(405)01 40 6 0.4058 0.8750 0.7325 0.7500 None SAM-M002-00(417)01 46 7 0.6215 0.5690 0.5065 0.1286 None SAM-M002-00(422)01 64 14 1.8148 1.1352 1.4016 1.8857 None SAM-M002-00(399)01 23 7 2.6650 1.5057 0.4957 0.3714 None STP-001-5(60)01 37 9 0.6994 0.7361 0.5000 0.5111 None STP-9-2(78)01 18 8 0.5776 2.5255 0.4667 0.8375 Variances STP-9-2(90)01 21 8 3.7786 2.7927 -0.0190 0.2125 None STP-34-1(22)01 89 14 2.5506 2.8623 -1.8416 -0.7071 None STP-36-1(13)01 49 15 10.8128 9.9617 0.5367 -0.1200 None STP-042-2(49)01 82 8 11.3187 21.1070 -0.0171 -2.4875 None 85 STP-141-1(7)01 36 21 7.9551 10.5381 0.6972 -0.0286 None STP-149-1(29)01 62 8 6.6796 9.2314 0.2065 0.9000 None STP-803(4)01 72 18 7.2362 5.5335 -0.2694 -0.6056 None STP-2434(2)01 14 6 0.7495 3.9710 0.6786 1.0500 Variances STP-M00-00(703)01 34 7 0.4411 0.2733 1.8794 1.7000 None STP-M001-00(273)01 82 19 1.4091 1.2050 -0.8329 -0.2053 None STP-M001-00(330)01 88 14 1.9975 0.8569 -1.2477 -0.9000 None STP-M001-00(490)01 25 6 2.4800 3.0297 0.7800 0.5167 None STP-M001-00(492)01 50 11 0.5925 0.6945 -0.8340 -0.8364 None STP-M001-00(493)01 97 10 3.0188 2.4827 0.4784 -0.0600 None STP-M001-00(773)01 76 8 2.1427 3.5621 -0.0408 0.3750 None STP-M001-00(872)01 77 6 1.7208 0.4867 -0.3909 0.2333 None STP-M002-00(093)01 45 7 0.7053 0.2348 1.1467 1.9143 None STP-M002-00(114)01 50 6 0.4072 0.4110 0.1240 -0.0500 None STP-M002-00(116)01 51 9 0.5617 1.1861 0.7471 0.8111 None STP-M002-00(154)01 62 9 0.8029 0.4453 -0.3065 -0.2556 None STP-M002-00(155)01 44 7 0.7935 4.5029 0.3000 -0.3571 Variances STP-M002-00(165)01 118 10 2.5686 2.3129 -1.2975 -2.1800 None STP-M002-00(169)01 32 6 1.7600 1.3747 -0.0750 0.5333 None STP-M002-00(175)01 122 18 1.0972 1.0406 -0.2893 -0.6944 None STPN-12-1(110)01 44 20 4.2579 4.9531 -1.9705 -2.1450 None STP-M002-00(395)01 47 11 1.0893 1.2420 -0.8979 -1.0000 None STP-M002-00(236)01 32 7 0.4322 0.3495 0.5531 0.4429 None STP-M002-00(397)01 93 16 0.0000 0.0000 0.0000 0.0000 None STP-M001-00(330)01 87 10 2.0204 1.7250 -1.2460 -1.5500 None STP-M002-00(115)01 73 10 2.9465 4.1610 -1.6658 -1.9100 None 86 Table C-2. ?? Project by Project Table for Comparison Tests Project # Nqct Ndot Vqct Vdot Mqct Mdot Differences CM-186-1(28)01 7 7 16.8528 28.6191 -1.1429 -2.4286 None EDS-27-(136)01 8 8 1.9498 2.2755 -0.4875 -0.5125 None EDS-27(147)01 9 9 7.1028 14.4894 -0.4444 0.3778 None EDS-84(16)01 7 7 11.4400 7.5295 1.5000 2.2429 None EDS-84(17)01 11 11 7.0185 5.2827 2.2364 2.2455 None EDS-555(6)01 7 7 14.9491 10.1029 -0.1714 0.0571 None GIP-341(34)01 14 14 18.8644 21.8710 2.3143 2.6214 None GIP-341(38)01 8 8 13.5041 16.6998 2.3875 3.2375 None IM-NH-85-2(148)01 11 11 43.3587 39.1025 2.9545 4.0364 None NH-026-2(81)01 10 10 1.7271 3.6222 -0.5600 0.6000 None NH-56-1(62)01 8 8 10.6227 23.7070 1.9625 1.5125 None NH-75-1(157)01 24 24 7.2089 8.4164 -0.2750 -0.2583 None NH-75-1(203)01 15 15 11.9492 16.7910 1.8067 1.3333 None NH-IM-75-1(158) 24 24 10.5087 8.1624 -0.1458 -0.0375 None NH-IM-95-1(119)01 7 7 3.2548 12.8995 -0.7143 0.2429 None NH-IM-95-1(125)01 15 15 3.8064 1.2511 -0.4933 -0.1400 None NH-IM-95-1(155)01 6 6 17.2667 7.4617 1.6667 4.6167 None NHS-M002- 00(398)01 26 26 7.0582 9.3768 0.2692 -0.3346 None NHS-M001- 00(530)01 20 20 14.3936 43.2898 2.8250 4.7350 None SAMA-39-(44)01 15 15 10.2470 7.2492 -0.3533 -0.9733 None SAMA-520(59)01 12 12 0.1715 0.7936 -0.2333 -0.5417 Variances STP-001-5(60)01 8 8 1.1543 0.3257 0.7000 0.3500 None STP-34-1(22)01 11 11 1.8096 1.2367 -2.0182 -1.1455 None STP-36-1(13)01 13 13 15.6133 9.4841 -1.2000 -1.6077 None STP-803(4)01 7 7 3.5462 6.1524 -0.8429 -0.0286 None STP-M001- 00(273)01 15 15 3.1084 1.8927 -0.6133 -0.4867 None STP-M001- 00(330)01 11 11 0.6900 2.0980 -1.5000 -1.0000 None STP-M001- 00(773)01 8 8 2.5021 4.4541 0.2750 0.5625 None STP-M002- 00(093)01 7 7 0.8357 1.0695 1.3714 1.3571 None STP-M002- 00(165)01 8 8 1.1841 8.8470 -1.0875 -0.6875 Variances STP-M002- 00(175)01 15 15 0.7812 1.4784 -0.2533 -0.0867 None STP-M002- 00(397)01 11 11 0.0000 0.0000 0.0000 0.0000 None STP-M001- 00(330)01 15 15 0.6554 1.7795 -1.6400 -1.1667 None STP-M002- 00(115)01 7 7 2.1824 0.5614 -1.8714 -1.2857 None 87 Table C-3. #200 Sieve Project by Project Table for QCT versus QA. Project # QC QA Vqc Vqa Mqc Mqa Differences BRN-42-2(40)01 15 6 0.0392 0.1787 0.9267 0.7333 None CM-186-1(28)01 38 7 0.6502 0.1090 0.1895 -0.8286 Means CSSTP-M002- 00(444)01 40 9 0.5246 0.8700 -0.1050 0.2667 None CSSTP-M002- 00(453)01 53 7 0.4101 0.9567 0.2792 0.4000 None EDS-27(136)01 140 35 0.3948 8.5014 1.0650 1.6086 None EDS-27(147)01 129 20 0.4037 0.3521 0.5605 0.6950 None EDS-84(16)01 129 16 0.4262 0.8333 0.0295 -0.3000 None EDS-84(17)01 63 11 0.3243 0.3380 -0.1222 0.0000 None EDS-545(8)01 72 9 0.1563 0.1011 -0.6583 -0.6111 None EDS-555(6)01 73 15 0.3415 0.5064 -0.6932 -0.4067 None EDS-565(7)-01 99 7 0.0871 0.5981 -0.3798 -0.1143 Variances EDS-565-(15)01 98 15 0.1185 0.4054 -0.3622 -0.0400 Variances FLF-540(8)01 61 8 0.5134 0.4850 0.3620 0.5750 None FLF-540(25)01 92 10 0.2787 0.3129 0.1435 0.1200 None FLF-540(28)01 116 10 0.2792 0.3877 -0.5802 -0.4100 None GIP-341(33)01 105 13 0.2229 0.2017 0.6114 0.4000 None GIP-341(34)01 141 25 0.8528 0.5892 0.6624 0.4560 None GIP-341(38)01 199 21 1.1388 0.6096 0.5005 0.7190 None HPP-3717-00(300)01 36 16 0.3483 6.6850 -0.2028 0.6313 Variances HPP-STP-178-1(33)01 23 7 0.3036 0.1429 0.9783 1.4429 None IM-00MS(268)01 43 14 0.5347 0.4699 0.2047 0.3071 None IM-285-1(349)01 29 8 0.3747 0.9457 0.6552 0.1500 None IM-0000-00(471)01 60 15 0.4679 0.4724 0.4300 0.2667 None IM-0000-00(472)01 98 16 0.6396 0.8358 0.1357 -0.1625 None IM-00MS(20)01 56 17 0.4090 1.2307 0.5607 0.6947 Variances IM-MS(325)01 31 7 0.4046 0.4162 0.8452 1.0429 None IM-NH-85-2(148)01 175 20 0.6567 0.6519 0.3851 0.2350 None LAR00-S005- 00(459)01 21 7 0.5155 0.4762 -0.4381 0.2429 None LAR00-S005- 00(593)C1 13 8 0.5286 1.2514 1.6769 1.0000 None LAR00-S005-00(636) 21 9 0.1613 0.0761 1.0857 1.1111 None LAR00-S005- 00(649)C1 13 6 0.2881 0.4857 1.2154 1.2167 None LAR00-S005- 00(569)C1 18 7 1.1331 0.1924 -0.3611 -0.5714 None LAR31-87-1(067)01 26 6 0.2552 3.9017 0.6654 -0.8833 Variances LAR32-2-3(067)01 17 6 0.4088 0.3057 0.2588 0.3167 None LAR32-2-5(63)C1 12 6 0.1663 0.5027 0.6083 0.1333 None LAR32-24-1(057)C1 43 8 0.4962 2.0813 0.4953 0.4125 Variances LAR32-30-2(089)01 34 9 0.7408 1.5775 1.1088 0.1667 None LAR32-85-1(77)C1 20 6 0.1887 0.4560 -0.5650 0.1000 Means LAR32-250-1(057)C1 37 8 0.2692 0.7364 0.1270 -0.0250 None LAR32-888-1(089)01 45 13 0.6898 0.9869 0.3556 -0.0769 None 88 LAU32-8532-79(121)1 28 10 0.8504 0.7499 0.5179 0.0900 None NH-11-1(44)01 34 11 0.8803 0.4045 1.0294 0.2364 None NH-012-1(83)01 59 10 0.2298 0.3529 1.2051 1.0200 None NH-017-2(53)01 36 13 0.7277 1.2076 0.6167 0.5615 None NH-20-2(183)01 240 35 0.6932 1.2740 0.0221 -0.1686 Variances NH-026-2(81)01 29 9 0.8698 0.1525 0.8138 0.8000 Variances NH-043-1(49)01 15 6 0.3600 1.1800 0.3000 -0.1000 None NH-56-1(62)01 53 17 0.2916 0.9456 0.1811 0.0059 Variances NH-75-1(157)01 234 30 1.1829 1.5977 0.3821 0.6467 None NH-75-1(203)01 178 18 0.4697 0.5379 0.4685 0.4556 None NH-171-1(4)01 100 8 0.7669 1.0584 0.4790 -0.0875 None NH-8013(8)01 64 20 0.2961 0.3768 0.4141 0.2000 None NH-IM-75-1(158) 174 24 0.1503 0.1039 -0.4598 -0.7708 Means NH-IM-95-1(119)01 94 14 0.3517 0.7376 -0.2553 -0.1286 None NH-IM-95-1(125)01 142 29 0.4681 0.6280 0.5923 0.6962 None NH-IM-95-1(155)-01 117 12 0.4071 2.0906 0.2256 -1.2833 Both NHS-0000-00(768)01 60 8 0.2364 0.3155 -0.5517 0.2875 Means NHS-M002-00(398)01 487 53 0.8156 1.4086 0.3357 0.1094 Variances NHS-M000-00(436)01 26 6 0.7331 1.5347 0.4115 0.5667 None NHS-M001-00(530)01 193 10 1.5599 0.9423 0.5508 0.2700 None NHS-M001-00(531)01 59 12 0.1680 0.4027 0.0915 0.1500 None NHS-M001-00(591)01 50 13 0.8509 1.0647 -0.3640 -0.4846 None NHS-M002-00(276)01 25 8 0.5574 0.4498 0.3920 0.3125 None NHS-MOO2- 00(292)01 248 14 0.6024 0.5181 0.6867 0.8500 None PR000-S005- 00(810)C1 18 7 0.2965 0.5014 1.2333 0.1857 Means PR-147-2(055)01 13 9 0.7323 1.2100 1.5692 1.5000 None PR-3-3(151)01 20 6 0.4992 0.4057 -0.0350 0.2167 None PR131-1(63)C1 8 8 0.1193 0.4279 0.3250 -0.5250 Means PR-1963-2(139)01 28 6 0.3417 0.1350 -0.6393 -0.3500 None PR-8530-83(39)C1 22 7 0.2492 0.5514 0.9818 0.0857 Means PR-8531-21(77)C1 50 10 0.3372 0.3107 0.3420 0.9800 Means PRLOP-8530- 32(15)C1 33 11 0.4761 0.8736 1.1212 0.9818 None SAMA-3(249)01 39 9 0.2983 5.2725 0.6897 -0.7333 Variances SAMA-3(294)01 123 25 0.2046 0.4732 0.6016 0.2640 Variances SAMA-3(298)01 238 31 0.6537 0.9971 1.0693 0.6226 Means SAMA-39(44)01 70 18 0.4312 0.4536 0.5314 0.4222 None SAMA-53(133)01 34 6 0.2912 0.1257 1.1971 0.8833 None SAMA-60(5)01 51 10 1.8202 1.6254 0.0020 -0.3100 None SAMA-74(56)01 26 9 0.4256 0.8944 0.0000 -0.5222 None SAMA-74(58)01 121 27 0.3228 0.2156 0.3521 0.4556 None SAMA91-(44)01 17 7 0.2328 0.1262 0.3176 0.4571 None SAMA-155(52)01 18 6 0.3272 0.1080 -0.2389 0.2000 None SAMA-206-CO(3)01 71 14 0.8426 1.4037 0.2169 0.1714 None SAMA-234-(17)01 51 8 0.2353 0.1964 0.2471 0.0250 None SAMA-520(58)01 84 18 0.3688 0.4296 0.8512 0.9389 None 89 SAMA-520(59)01 144 21 0.4100 0.7153 0.9639 0.6333 None SAM-M002-00(405)01 40 6 0.4567 0.3870 0.0350 0.3500 None SAM-M002-00(417)01 46 7 0.5425 0.7162 0.5130 -0.0429 None SAM-M002-00(422)01 64 17 0.3435 0.1803 1.1063 0.7176 None SAM-M002-00(399)01 23 7 0.3598 1.1314 0.8565 0.5143 None STP-001-5(60)01 37 9 0.0835 0.1050 0.6378 0.7333 None STP-9-2(78)01 29 12 0.2969 0.4627 1.1759 1.2500 None STP-9-2(90)01 22 8 0.7004 0.9312 0.9682 0.5375 None STP-34-1(22)01 89 14 0.3345 0.6146 -0.2809 -0.1071 None STP-36-1(13)01 52 15 0.3473 0.5112 1.3827 1.1133 None STP-042-2(49)01 81 8 0.4149 0.5684 0.7741 0.7375 None STP-141-1(7)01 36 21 0.2936 1.6669 0.7111 -0.5905 Both STP-149-1(29)01 62 8 0.1694 1.1993 0.4677 -0.0250 Variances STP-803(4)01 72 18 0.4468 0.1697 1.0861 1.3556 None STP-2434(2)01 11 6 0.0720 1.8657 -0.1000 0.1167 Variances STP-4067(2)02 96 7 0.2505 0.1562 0.6521 0.6429 None STP-5121(3)01 34 7 0.2820 0.5433 0.5912 0.6000 None STP-M00-00(703)01 54 9 0.8410 1.3644 -0.1593 -0.0222 None STP-M001-00(273)01 102 18 0.3875 0.4352 0.5863 0.4889 None STP-M001-00(490)01 24 6 0.7910 0.3657 0.8167 0.5833 None STP-M001-00(492)01 54 13 0.4932 0.1023 0.0130 0.2692 Variances STP-M001-00(493)01 121 13 0.3546 0.3967 -0.9694 -0.7000 None STP-M001-00(773)01 76 9 0.4820 0.6175 0.5737 0.4667 None STP-M001-00(872)01 77 6 0.3587 0.4587 0.2390 0.2667 None STP-M002-00(92)01 44 8 1.0007 1.4827 -0.0932 -0.3625 None STP-M002-00(093)01 45 7 0.2751 0.2048 0.8889 0.6143 None STP-M002-00(114)01 60 6 0.2947 0.0707 0.2083 -0.0333 None STP-M002-00(116)01 51 11 0.1184 0.0756 -0.4608 -0.0818 Means STP-M002-00(154)01 80 13 0.5608 0.4933 0.7650 0.5000 None STP-M002-00(155)01 44 7 0.3935 0.4514 0.2000 0.3857 None STP-M002-00(165)01 118 10 0.3073 0.5934 0.2093 -0.3300 Means STP-M002-00(169)01 32 6 0.4019 0.4120 1.0250 1.1000 None STP-M002-00(173)01 20 7 0.2333 0.2124 1.1800 0.7286 None STP-MOO2-00(175)01 122 18 0.1545 0.1375 0.5992 0.5889 None STP-M002-00(232)01 33 12 1.0950 1.8075 0.3242 -0.2750 None STP-M002-00(236)01 32 7 0.3646 0.2790 1.0156 1.1286 None STP-M002-00(397)01 93 16 0.5557 3.2172 0.6430 0.6625 None STP-M001-00(330)01 88 24 0.2392 0.4448 0.1057 -0.2042 None STP-M002-00(115)01 73 10 0.1716 0.1246 -0.0849 -0.0700 None STP-M002-00(395)01 68 13 0.7418 0.4100 0.3103 0.3000 None STPN-12-1(110)01 45 20 0.5239 0.7803 0.8200 0.6850 None 90 Table C-4. #200 Sieve Project by Project Table for Comparison Tests. Project # Nqc Ndot Vqc Vdot Mqc Mdot Differences CM-186-1(28)01 7 7 1.1767 0.4124 -0.2000 -0.8714 None CSSTP-M002- 00(444)01 6 6 0.4907 1.1177 -0.2667 -0.1167 None EDS-27(136)01 6 6 0.1830 0.2840 0.9500 0.9000 None EDS-27(147)01 16 16 0.2543 0.3730 0.3313 -0.1312 None EDS-84(16)01 6 6 0.0987 0.2297 -0.2333 -0.1167 None EDS-84(17)01 11 11 0.3342 0.4327 -0.1273 -0.2545 None EDS-555(6)01 7 7 0.1329 0.1614 -0.9571 -0.7143 Means GIP-341(34)01 14 14 0.5305 0.6394 0.7857 0.6357 None GIP-341(38)01 8 8 0.9164 0.7971 1.0750 1.1000 None IM-MS(325)01 6 6 0.1870 0.6058 1.0500 1.2250 None IM-NH-85-2(148)01 11 11 1.1565 3.5836 1.0636 1.3182 None NH-20-2(183)01 7 7 0.5357 0.8162 -0.5286 -0.4429 None NH-026-2(81)01 10 10 0.8588 1.5329 0.7900 0.4200 None NH-56-1(62)01 8 8 0.2427 0.6784 0.0375 0.0875 None NH-75-1(157)01 22 22 0.8371 0.8606 0.5909 0.4591 None NH-75-1(203)01 15 15 0.5270 0.5735 0.6133 0.3933 None NH-IM-75-1(158) 24 24 0.2786 0.3409 -0.4067 -0.6000 None NH-IM-95-1(119)01 7 7 0.1029 1.0857 -0.2571 0.6286 Variability NH-IM-95-1(125)01 15 15 0.5464 1.9098 0.6933 1.0533 None NH-IM-95-1(155)01 10 10 0.3600 2.1329 0.1000 -0.7800 Variability NHS-M002-00(398)01 25 25 0.5207 1.2893 0.5640 0.3480 None NHS-M001-00(530)01 19 19 1.3956 5.1427 0.1000 0.3053 None NHS-M002-00(294)01 12 12 0.3481 0.9661 0.8917 -0.1333 Means PR-128-2(065)01 6 6 0.4720 0.3257 0.9000 0.9167 None SAMA-39(44)01 15 15 0.4210 0.3070 0.6333 0.4467 None SAMA-45(37)01 6 6 0.1977 0.6817 1.3167 2.0833 None SAMA-520(59)01 12 12 0.2499 0.5130 0.7917 1.0250 None STP-001-5(60)01 8 8 0.1827 0.1400 0.5625 0.1000 None STP-9-2(78)01 6 6 1.7617 3.8827 0.7167 -0.1333 None STP-34-1(22)01 11 11 0.4016 0.5000 -0.3182 -0.4000 None STP-36-1(13)01 13 13 1.5191 0.6547 1.0077 1.3154 None STP-803(4)01 7 7 0.2533 1.1114 1.2000 1.0143 None STP-911(7)01 6 6 0.8987 0.1817 0.3333 0.3833 None STP-M00-00(703)01 7 7 1.1781 1.1733 0.4143 -0.4000 None STP-M001-00(273)01 15 15 0.5150 0.2946 0.5267 0.7200 None STP-M001-00(773)01 7 7 0.4233 0.4029 1.0000 0.6571 None STP-M002-00(093)01 7 7 0.2562 0.5362 0.9571 0.7571 None STP-M002-00(114)01 6 6 0.1787 0.2937 -0.0667 -0.3167 None STP-M002-00(165)01 8 8 0.1927 0.2612 0.3125 0.0875 None STP-M002-00(175)01 15 15 0.1012 0.6397 0.5533 0.2600 Variability STP-M002-00(397)01 11 11 0.2227 0.9502 0.5545 -0.0727 None STP-M001-00(330)01 11 11 0.1849 0.3567 0.0091 -0.2455 None STP-MOO2-00(115)01 9 9 0.0961 0.1400 -0.0889 -0.0333 None 91 STP-MOO2-00(252)01 6 6 6.0297 4.0067 -1.0167 -0.8333 None STP-MOO2-00(395)01 6 6 0.6257 1.5240 0.3167 0.6000 None 92 Table C-5. Asphalt Content Project by Project Table for QCT versus QA. Project # QC QA Vqc Vqa Mqc Mqa Differences BRN-42-2(40)01 13 6 0.0083 0.0168 0.0569 0.0483 None CM-186-1(28)01 38 6 0.0370 0.0204 0.0021 0.1550 None CSSTP-M002-00(444)01 36 9 0.0217 0.0421 0.0011 -0.1222 None CSSTP-M002-00(453)01 53 7 0.0159 0.0337 0.0225 -0.0457 None EDS-27(136)01 138 32 0.0159 0.0176 0.0258 0.0094 None EDS-27(147)01 130 20 0.0102 0.0196 -0.0417 -0.0030 None EDS-545(8)01 72 9 0.0309 0.0183 0.0275 0.1522 None EDS-555(6)01 73 15 0.0384 0.1352 0.0403 0.2260 Variances EDS-565-(15)01 98 15 0.0364 0.0301 0.0040 0.0000 None EDS-565-(7)01 99 7 0.0233 0.0330 -0.0420 -0.0343 None EDS84(16)01 130 16 0.0211 0.0298 0.0192 -0.0888 Mean EDS-84(17)01 65 11 0.0159 0.1757 0.0809 0.1364 Variances FLF-540(25)01 95 13 0.0189 0.0494 0.0002 0.0315 Variances FLF-540(28)01 116 10 0.0317 0.0295 0.0391 0.1580 None FLF-540(8)01 61 8 0.0250 0.0135 0.0816 -0.0300 None GIP-341(33)01 107 13 0.0248 0.0209 0.0183 -0.1062 Mean GIP-341(34)01 145 25 0.1082 0.1254 0.1427 0.0468 None GIP-341(38)01 203 21 0.0845 0.1516 -0.0118 -0.0029 None HPP-3717-00(300)01 36 16 0.0286 0.0355 -0.0653 -0.0413 None HPP-STP-178-1(33)01 23 7 0.0363 0.0038 -0.2222 -0.1800 Variances IM-0000-00(471)01 57 15 0.0396 0.0498 0.0974 0.0660 None IM-0000-00(472)01 73 10 0.0443 0.0812 0.0164 0.0230 None IM-00MS(20)01 56 15 0.0546 0.0852 -0.0636 0.0220 None IM-00MS(268)01 41 14 0.0231 0.0472 0.0463 0.0214 Mean IM-285-1(349)01 30 8 0.0331 0.1437 -0.0077 0.1963 Variances IM-MS(325)01 31 7 0.0200 0.0115 -0.0710 0.0757 None IM-NH-85-2(148)01 175 20 0.0674 0.1105 -0.0145 0.0080 None LAR00-S005- 00(459)01 21 7 0.0336 0.2947 -0.0795 -0.0186 Variances LAR00-S005- 00(593)C1 13 8 0.0689 0.0667 -0.0985 -0.1687 None LAR00-S005- 00(649)C1 13 6 0.0728 0.0382 -0.0062 -0.4367 Mean LAR00-S005- 00(666)C1 8 7 0.0532 0.0372 0.0825 -0.0200 None LAR31-87-1(067)01 25 7 0.0181 0.0485 -0.0904 -0.0157 None LAR32-2-3(067)01 18 6 0.0357 0.0412 -0.0661 0.0333 None LAR32-888-1(089)01 44 12 0.0583 0.0312 -0.0925 -0.0917 None LAU32-8532- 79(121)01 16 6 0.0339 0.1333 0.2031 0.2667 None NH-012-1(83)01 60 10 0.0353 0.0256 -0.1027 -0.1170 None NH-017-2(53)01 36 13 0.0464 0.0726 -0.0711 0.0915 None NH-026-2(81)01 29 9 0.0420 0.0417 -0.0300 -0.0456 None NH-043-1(49)01 15 6 0.0462 0.0339 0.1200 0.0232 None NH-11-1(44)01 34 10 0.0732 0.0590 0.0921 -0.0550 None 93 NH-171-1(4)01 100 8 0.0448 0.0351 -0.0253 -0.0500 None NH-20-2(183)01 240 29 0.0326 0.0519 -0.0117 0.0179 None NH-56-1(62)01 53 16 0.0787 0.1061 0.0721 -0.1400 None NH-75-1(157)01 234 29 0.0584 0.0438 0.0233 0.0093 None NH-75-1(203)01 180 18 0.0223 0.0314 -0.0245 0.0022 None NH-8013(8)01 64 15 0.0385 0.0303 0.0587 0.0353 None NH-IM-75-1(158) 174 21 0.0236 0.0186 -0.0389 -0.0495 None NH-IM-95-1(119)01 90 14 0.0303 0.0449 0.0006 -0.0493 None NH-IM-95-1(125)01 142 29 0.0210 0.0223 -0.0182 -0.0152 None NH-IM-95-1(155)-01 117 12 0.0538 0.0561 0.0549 0.0883 None NHS-0000-00(768)01 61 8 0.0330 0.0672 0.0631 0.0763 None NHS-M001-00(530)01 193 9 0.0499 0.0667 -0.0331 -0.0756 None NHS-M001-00(531)01 59 12 0.0291 0.0241 -0.0478 -0.0733 None NHS-M001-00(591)01 51 13 0.0554 0.0584 -0.0816 -0.0469 None NHS-M002-00(276)01 25 8 0.0224 0.0258 -0.0156 0.1525 None NHS-M002-00(292)01 251 10 0.0425 0.0453 0.0615 0.0040 None NHS-M002-00(398)01 487 53 0.0388 0.0613 -0.0556 -0.1036 Variances PR-147-2(055)01 13 9 0.0698 0.1396 0.0223 -0.0133 None PR-1963-2(139)01 28 6 0.0678 0.0768 0.0239 -0.0400 None PR-3-3(151)01 21 6 0.0521 0.0305 0.0614 -0.0267 None PR-8530-83(39)C1 22 10 0.0327 0.0500 -0.0873 0.0960 None PR-8531-21(77)CT1 49 10 0.0482 0.0981 -0.0059 0.0810 None PRLOP-8530- 32(15)C1 31 11 0.0375 0.0339 0.0587 -0.0236 None SAMA-155(52)01 18 6 0.0134 0.0294 -0.0983 -0.1617 None SAMA-206-C0(3)01 70 14 0.0504 0.0327 0.0271 0.1279 None SAMA-234-(17)01 51 8 0.0111 0.0058 -0.0553 -0.0500 None SAMA-3(249)01 38 9 0.0264 0.1858 0.0618 0.1167 Variances SAMA-3(294)01 123 25 0.0123 0.0551 0.1577 0.1488 Variances SAMA-3(298)01 229 31 0.0579 0.0740 0.0089 -0.0258 None SAMA-39(44)01 70 18 0.0393 0.0218 -0.0560 -0.1294 None SAMA-520(58)02 84 18 0.0255 0.0280 0.0692 -0.0117 None SAMA-520(59)01 144 21 0.0405 0.0416 0.1119 0.0352 None SAMA-53(133)01 34 6 0.0175 0.0062 -0.0729 -0.0350 None SAMA-60(5)01 50 10 0.0546 0.0628 -0.0544 -0.0580 None SAMA-74(56)01 25 9 0.0346 0.0646 0.0312 0.0911 None SAMA-74(58)01 121 27 0.0402 0.0089 -0.0235 -0.0867 Variances SAM-M002-00(399)01 23 7 0.0381 0.0922 -0.0026 0.1200 None SAM-M002-00(405)01 40 6 0.0202 0.0426 0.0015 -0.0550 None SAM-M002-00(417)01 46 7 0.0209 0.0386 0.0654 -0.1143 Mean SAM-M002-00(422)01 64 17 0.0261 0.1004 0.0548 0.1988 Variances STP-001-5(60)01 37 9 0.0367 0.0196 -0.0895 0.0267 None STP-042-2(49)01 82 8 0.0386 0.0438 0.1521 0.0100 None STP-141-1(7)-01 36 21 0.0274 0.0346 0.0008 0.0524 None STP-149-1(29)01 62 8 0.0454 0.1082 0.0218 -0.1037 None STP-2434(2)01 14 6 0.0161 0.0208 0.0814 0.0867 None STP-34-1(22)01 89 14 0.0159 0.0192 0.0201 0.1157 None STP-36-1(13)01 49 15 0.0531 0.0564 -0.0463 -0.0660 None 94 STP-4067(2)02 98 7 0.0275 0.0251 0.0756 0.1243 None STP-5121(3)01 34 7 0.0088 0.0094 0.1921 0.2071 None STP-803(4)01 73 17 0.0222 0.0238 -0.0025 0.0135 None STP-9-2(78)01 29 12 0.0120 0.0221 -0.0293 0.0258 None STP-9-2(90)01 22 8 0.0221 0.0205 0.0732 0.1100 None STP-M00-00(703)01 53 9 0.0208 0.0257 0.0740 -0.0056 None STP-M001-00(273)01 100 18 0.0298 0.0227 0.0087 -0.1083 Mean STP-M001-00(330)01 88 14 0.0437 0.0318 0.0102 -0.0286 None STP-M001-00(490)01 25 6 0.1374 0.1993 0.1120 0.1667 None STP-M001-00(492)01 54 13 0.0125 0.0160 -0.0256 -0.0508 None STP-M001-00(493)01 121 13 0.0185 0.0315 -0.0264 -0.0169 None STP-M001-00(773)01 76 9 0.0175 0.0438 0.0080 0.0344 None STP-M001-00(872)01 77 7 0.0164 0.0225 0.0522 0.1257 None STP-M002-00(093)01 45 7 0.0075 0.0206 0.0602 0.1200 None STP-M002-00(114)01 60 6 0.0109 0.0267 0.0560 0.0583 None STP-M002-00(115)01 73 10 0.0088 0.0155 -0.0026 -0.0070 None STP-M002-00(116)01 51 11 0.0111 0.0109 0.0408 0.0409 None STP-M002-00(154)01 80 13 0.1161 0.3721 0.0769 0.0223 Variances STP-M002-00(155)01 44 7 0.0212 0.0396 0.0116 -0.1457 None STP-M002-00(165)01 118 10 0.0089 0.0142 0.0168 -0.0350 None STP-M002-00(169)01 32 6 0.0157 0.0241 -0.1112 0.0583 Mean STP-M002-00(173)01 20 7 0.0223 0.0559 0.0745 0.0514 None STP-M002-00(232)01 33 12 0.0652 0.0598 -0.0370 0.0317 None STP-M002-00(236)01 32 7 0.0315 0.0210 0.1131 0.1000 None STP-M002-00(395)01 68 13 0.0414 0.0199 -0.0060 0.1046 None STP-M002-00(397)01 93 15 0.0240 0.0230 -0.0260 0.1087 Mean STP-M002-00(92)01 44 8 0.0315 0.0970 -0.1477 -0.0813 None 95 Table C-6. Asphalt Content Project by Project Table for Comparison Tests. Project # Nqc Ndot Vqc Vdot Mqc Mdot Differences CM-186-1(28)01 7 7 0.0518 0.0861 -0.0886 -0.0371 None CSSTP-M002-00(444)01 6 6 0.0218 0.0603 -0.0983 -0.3000 None EDS-27-(136)01 8 8 0.0231 0.0600 0.0038 -0.0062 None EDS-27(147)01 16 16 0.0149 0.0345 0.0044 0.0762 None EDS-84(16)01 7 7 0.0344 0.0370 0.0371 0.0071 None EDS-84(17)01 11 11 0.0201 0.0085 0.1382 0.0918 None EDS-555(6)01 7 7 0.1698 0.2422 0.1357 0.1857 None GIP-341(34)01 14 14 0.1124 0.1143 0.2129 0.1757 None GIP-341(38)01 8 8 0.0593 0.1549 0.1225 0.1350 None IM-MS(325)01 6 6 0.0153 0.0566 -0.0467 -0.2750 None IM-NH-85-2(148)01 11 11 0.2703 0.3765 0.1336 0.2991 None NH-20-2(183)01 7 7 0.0534 0.0363 -0.0257 -0.0757 None NH-56-1(62)01 8 8 0.0946 0.1336 -0.0113 0.0025 None NH-75-1(157)01 24 24 0.0362 0.0942 -0.0092 0.0250 None NH-75-1(203)01 18 18 0.0230 0.0299 -0.0183 0.0128 None NH-IM-75-1(158) 24 24 0.0455 0.0591 -0.1304 -0.0933 None NH-IM-95-1(119)01 7 7 0.0142 0.0561 -0.0486 0.0814 None NH-IM-95-1(125)01 15 15 0.0197 0.0663 0.0060 -0.0507 None NH-IM-95-1(155)-01 10 10 0.0447 0.0442 0.0650 0.1710 None NHS-M002-00(398)01 29 29 0.0374 0.0773 -0.1055 -0.1072 None NHS-M001-00(530)01 18 18 0.0713 0.2855 0.0811 0.1372 Variances PR-8530-83(39)C1 6 6 0.0489 0.0527 -0.0050 -0.0100 None SAMA-39-(44)01 15 15 0.0072 0.0262 -0.0447 -0.1393 None SAMA-206-CO(3)01 9 9 0.0393 0.1971 0.1389 0.3544 None SAMA-520(59)01 12 12 0.0529 0.0514 -0.0025 0.0383 None STP-037-2(60)01 11 11 0.1184 0.2392 -0.1364 -0.0755 None STP-9-2(78)01 6 6 0.0179 0.0190 -0.0683 -0.0600 None STP-34-1(22)01 11 11 0.0214 0.0368 0.0055 0.0164 None STP-36-1(13)01 13 13 0.0525 0.1435 -0.0008 0.1223 None STP-803(4)01 7 7 0.0305 0.0742 -0.0629 -0.1957 None STP-M00-00(703)01 7 7 0.0270 0.0287 0.0529 -0.0471 None STP-M001-00(273)01 15 15 0.0156 0.0344 -0.0167 -0.1027 None STP-M001-00(330)01 11 11 0.0643 0.0340 0.0691 0.0736 None STP-M001-00(773)01 8 8 0.0100 0.0377 0.0675 0.0750 None STP-M002-00(093)01 7 7 0.0199 0.0128 0.0571 0.0986 None STP-M002-00(114)01 6 6 0.0111 0.0928 0.0383 0.0000 None STP-M002-00[116]01 6 6 0.0063 0.0078 0.1000 0.0783 None STP-M002-00(165)01 8 8 0.0063 0.0093 0.0437 -0.0250 None STP-M002-00(175)01 15 15 0.0229 0.0440 0.0140 0.0920 None STP-M002-00(397)01 11 11 0.2125 0.2669 -0.1718 0.0227 Means STP-M002-00(115)01 7 7 0.0031 0.0200 -0.0257 -0.0357 None 96 APPENDIX D PROJECT BY PROJET ANALYSIS SCATTERPLOTS 0 5 10 15 20 25 30 35 40 45 2040608010120 QA QC T Figure D-1. Project by Project Variances for ?? Sieve QCT versus QA. -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 Average QA - JMF (% Passing) A ver a g e Q C - JMF (% P asg g i n g ) Linear (Line of Absolute Numerical Equality) Figure D-2. Project by Project Average Means for ?? Sieve QCT versus QA. 97 0 5 10 15 20 25 30 35 40 45 50 0 5 10 15 20 25 30 35 40 45 50 DOT QC T Figure D-3. Project by Project Variances for ?? Sieve for Comparison Tests. -5 -4 -3 -2 -1 0 1 2 3 4 5 -5 -4 -3 -2 -1 0 1 2 3 4 5 Average DOT-JMF (% Passing) A v e r ag e Q C T - J M F ( % P assi n g ) Figure D-4. Project by Project Means for ?? Sieve for Comparison Tests. 98 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 0123456789 QA QC Figure D-5. Project by Project Variances for #200 Sieve QCT versus QA. -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 Average QA-JMF (% Passing) A v e r ag e Q C -JMF ( % P a ssi n g ) Figure D-6. Project by Project Average Means for #200 Sieve QCT versus QA. 99 0 1 2 3 4 5 6 7 1234567 QA QC T Figure D-7. Project by Project Variances for #200 Sieve Comparison Tests. -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 Average DOT-JMF (% Passing) A v e r a g e QC T- J M F ( % Pa s s i n g) Figure D-8. Project by Project Average Means for #200 Sieve Comparison Tests. 100 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 QA QC T Figure D-9. Project by Project Variances for Asphalt Content QCT versus QA. -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 Average QA-JMF (% Passing) A v er a g e Q C - J MF (% Pa s s in g ) Figure D-10. Project by Project Average Means for AC for QCT versus QA. 101 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 DOT QC T Figure D-11. Project by Project Variances for AC for Comparison Tests. -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 Average DOT-JMF (Asphalt Content) Av e r ag e Q C T (As p h a l t C o n t en t) Figure D-12. Project by Project Average Means for AC for Comparison Tests.