Final+Exam+EDF6938


 * Quantitative Stats I - Final Exam**


 * ====**External Validity (generalization)**====

=
the extent to which research findings generalize to other populations, other times, or other settings (p. 197); how well does the relationship between two variables generalize across settings, samples, and times====

“//If not, we need to limit our claims to the people or setting studied and to ask what unique factors help account for the results.”//
||
 * ====Interaction of Selection and Treatment==== || ====• occurs when the treatment effects only generalize to those who are selected in the same way as the present sample====

• can increase and support wider generalizability if include a variety of subjects in a single study, include samples from different populations or larger populations
Ways to avoid threat [from lecture]:
 * Random selection
 * Ensure participation is convenient ||
 * ====Interaction of Setting and Treatment==== || ====• when the treatment effects generalize only to those settings used in the study====

• if you can’t replicate, must limit generalization claims
[from lecture]: Solution is to vary the settings and to analyze the independent variable/dependent variable relationship within each. ||
 * ====Interaction of History and Treatment==== || ====• occurs when the treatment effects generalize only to particular times, past or future====

• major public events from a time period can affect a study as do “special’ days
[from lecture]: Solutions include replicating the study at different times or conducting a review of the literature to establish whether or not prior evidence refutes the existence of the relationship. ||
 * ====**Population vs. Sample (p. 126 – 130)**====
 * ====**Population vs. Sample (p. 126 – 130)**====

subset of individuals selected from a larger population
[from lecture] Population--the complete set of observations about which we draw conclusions Experimentally accessible population-subset of population; a list of everyone in the population who is available for the study and has a good chance of being included in the sample Sample--actual observations included in the study

Sample and Population are not interchangeable; you cannot have a "sample population" ||
 * ====Sample Bias==== || ====• when numerous samples are on average underepresentative of the population because they miss important elements from those selected==== ||
 * ====Sampling Error==== || ===== a measure of how well a sample approximates the characteristics of a population====

• the higher the sampling error, the less precision one has in sampling and the more difficult it will be to make a case that the sample generalizes to the population
||
 * ====Cluster Sampling==== || ===== when sampling unit consists of a group (i.e. colleges) rather than individuals====


 * can be probabilistic or nonprobabilistic
 * if groups vary in size, random selection may underrepresent larger groups
 * for groups of varying size, use Probability Proportionate to Size to provide more accurate weighting for individuals ||
 * ====Probability Sampling==== || ====• helps reduce sample bias; also helps with sample error====

all members of a population have an equal chance of probability of being selected
||
 * ====Random Sampling==== || ===== drawing a representative group from a population by a method that gives every member of the population an equal chance of being drawn or selected==== ||
 * ====Systematic Sampling==== || ====• type of sampling that usually occurs in practice====

draws every nth person from an existing list that began as randomly chosen
Used instead of random sampling and provides results similar to that of random sampling ||
 * ====Stratified Sampling==== || ===== when elements of a population are grouped or classified together based on a certain category or common characteristic (i.e. male, female), separate samples are then selected from each strata; final sample is a combination of all stratas====
 * ====//Probability proportionate to size – equal probability of selection when the sizes of the population varies (i.e. 10 out of 100; 100 out of 1000)//====
 * ====//disproportionate (weighting) – adjusting the sample so that one group has greater chance of selection over another//====

It is important to understand the significance of the strata or subpopulation. ||
 * ====Nonprobability Sampling (p 129)==== || ===== any method of sampling in which the elements have unequal chances of being selected====
 * ====//convenience sampling = completely dependent on the availability of the respondents//====
 * ====//purposive sampling = respondents are chosen because they meet a certain criteria or match a certain characteristic//====
 * ====//quota sampling = try to create a sample that matches a predetermined demographic profile//==== ||
 * ====Disproportionate Sampling==== || ===== occurs when a subpopulation is over-sampled or under-sampled====

• sample proportion differs from population
Reasons for sampling in different proportions:
 * To guarantee the group is sampled in sufficient numbers for analysis
 * May wish to subdivide cases within stratum for further analysis ||


 * ====**Statistical Conclusion Validity**====

• Are the statistics and the conclusions we draw from them reasonable?
====• Three issues to consider – sensitivity (power), significance (Type I), effect size [NOTE: From the PPTs, these issues are Sensitivity (power); Evidence of covariation (Type I), and Strength of the relationship (effect sizes -- how strong is the relationship between variables?)==== ||
 * ====Sensitivity (Power)==== || ====Power refers to the probability that your test will find a statistically significant difference when such a difference actually exists. In other words, power is the probability that you will reject the null hypothesis when you should (and thus avoid a Type II error). It is generally accepted that power should be .8 or greater; that is, you should have an 80% or greater chance of finding a statistically significant difference when there is one.====

Ways to increase power:

 * ====increase effect size (Treatment that produces a bigger effect is easier to identify. Small effect size requires larger sample.)====
 * ====increase Type I error rate (using higher rate - .10 vs. .05 improves effect, but not something you usually want to adjust. May produce more errors.)====
 * ====decrease error variance (smaller error variance produces more power. Don’t want to restrict range of scores. Better to use more powerful statistics.)====
 * ====increase sample size (bigger numbers give more stable estimates and more reliable conclusion – larger studies have better chance of finding difference)==== ||
 * ====Error Variance==== || ===== square of the standard deviation====

• looks at the variance accounted for by factors missed out in the study
||
 * ====Null Hypothesis==== || ====HO – means no difference or no relationship; acts as a benchmark against which actual outcomes of a study can be measured====

• indirectly tested
||
 * ====Research Hypothesis==== || ===== a definite statement that there is a relationship between variables====

//directional – reflects a difference between groups, and the direction is specified (one-tailed test)//
||
 * ====Evidence of Covariation (Type I) - Significance==== || ====//Type I Error// = is made when the researcher decides to reject the null hypothesis when the null is true; typically set at .05====

//Type II Error// = is made when the research “accepts” the null hypothesis when the null if false
||
 * ====Statistical Significance==== || ==== • chance that the changes you observe in your participants’ knowledge, attitudes, and behaviors are due to chance rather than to the program ====

==== To learn whether the difference is statistically significant, you will have to compare the probability number you get from your test (the p-value) to the critical probability value you determined ahead of time (the alpha level). ==== http://horan.asu.edu/cook&campbell.htm || http://horan.asu.edu/cook&campbell.htm ||
 * ==== If the p-value is less than the alpha value, you can conclude that the difference you observed is statistically significant. ==== ||
 * ====P-Value==== || ==== = the probability that the results were due to chance and not based on your program. ====
 * ==== P-values range from 0 to 1. ====
 * ==== The lower the p-value, the more likely it is that a difference occurred as a result of your program. ==== ||
 * ====Alpha Level==== || ==== = the error rate that you are willing to accept ====
 * ==== Usually set at .05 or .01.also known as the Type I error rate ====
 * ==== alpha of .05 means that you are willing to accept that there is a 5% chance that your results are due to chance rather than to your program. ==== ||
 * ====Fishing==== || ===== occurs when more than one comparison is made within a study==== ||
 * Reliability of Treatment || Measures of low reliability may not register true changes.
 * Random Irrelevancies in Setting || Setting variables may divert respondents' attention to the treatment and/or introduce error variance, thus washing out treatment effects.


 * ====**Statistical Procedures**==== ||
 * ====Descriptive Statistics==== || ===== used to organize and describe the characteristics of a collection of data; don’t always refer back to the population – used to paint a picture (tells us what the participants look like)==== ||
 * ====Inferential Statistics==== || ===== tools that are used to infer the results based on a sample to a population - based on relationships between variables==== ||
 * ====Scales of Measurement==== || ====Scale used in study dictates what type of statistical measure can be used.====
 * ====nominal/categorical – fit one and only one class or category (i.e. female/male) – names – least precise measure (deals with frequencies, numbers)====
 * ====ordinal – things being measured are ordered (i.e. rank); rarely reported, but also uses frequencies (e.g., percent of students with 3 or higher on FCAT)====
 * ====interval – based on an underlying continuum (i.e. test, likert scale)====
 * ====ratio – has absolute 0 (test or survey data)====
 * interval & ratio describe center of distribution and standard deviation ||
 * ====Bivariate==== || ====• When looking at two variables and their relationships at the same time====

• Usually interval/ratio or nominal (interval with nominal = compares group means; nominal w/ nominal = looks at crosstabulations)
====• Most common stats is correlational to show strength of relationship (+1 to -1); plus indicates as one var increases, so does the other; negative means as one var increases, the other decreases. Weaker relationship is defined by 0.==== ||
 * ====Statistical Adjustment==== || ====- Choosing the right statistical procedure/tool to do the job==== ||


 * ====**Types of Research**==== ||
 * ====Experimental (Quantitative)==== || ===== looks at the effectiveness of programs or treatments; compares treatment effects====

• strongest design includes random assignment of control groups

 * ====True experimental – high internal validity; strongest for causal claim; control group with random assignment====
 * ====Quasi-experimental – moderate for causal claims; includes control group but without random assignment====
 * ====Pre-experimental – low internal validity; weak for causal claims; does not include control group====

• external validity can be weak due to strong experimenter control
||
 * ====Survey (Quantitative)==== || ===== a collection data from a population using a questionnaire; purpose is to describe a population====
 * ====//longitudinal// – data collected over time with two or more data collections (i.e. panel, trend, cohort)====
 * ====//cross-sectional// – data collected at one point in time====

• internal validity not a real issue
also - statistical conclusion validity - based on descriptive statistics and & standard errors. ||
 * ====Correlational (Quantitative)==== || ===== looks at the relationships of variables in natural settings; trying to control for what is not under the control of the experimenter====

Types of data collection:
 * longitudinal - data collection occurs over time
 * trend: different samples from changing population
 * cohort: different samples from same population
 * panel: follows same individuals over time
 * Cross-sectional - all data collection occurs at a certain point in time

• usually has the most complete statistics
|| Internal validity is the primary importance; focus is on outcomes of programs
 * ====Evaluation (Quantitative)==== || ===== examines the quality of local programs====

• usually is linked to multiple stakeholders with multiple questions

 * ====//product evaluation//====
 * ====//process evaluation//====
 * ====//accreditation//====
 * ====//needs assessment//====

• external validity not as important
||

 ¡ Ways to avoid threat  l Random selection l Ensure participation is convenient