The Open Educational Resources (OER) movement was started roughly ten years old (Wiley & Gurell, 2009). Since that time thousands of resources have been produced. Though these resources have been used both for classroom development and for the autodidact, the development of OER was not without problems. Incompatibility between Creative Commons licenses has made revising and remixing two resources difficult, if not impossible (Linksvayer, 2006). Tools to help educators find appropriate educational resources have been necessary but are still nascent. Educators' perceived quality issues have also hampered adoption (Wiley & Gurell, 2009). The result is that resources were only being minimally reused (Wiley, 2009). One possible reason observed for the limited reuse was the barrier of technology. Some resources were easier to view, revise and remix from a technical perspective than others. Hilton, Wiley, Stein, and Johnson (2010) created the ALMS analysis framework to assess the technical openness of an OER. Although the ALMS framework allowed for an assessment of OER, no pilot instrument was reported in the Hilton et al. (2010) article. The framework has not been tested because there is no known rubric with which measurement can occur. Consequently, Hilton et al.'s framework needed to be further developed and tested against a range of open educational resources. This dissertation examined the ALMS analysis, which was previously only a concept, in order to create a concrete framework with sufficient detail and documentation for comparisons to be made among OERs. The rubric was further refined through a Delphi study consisting of experts in the field of OER (n=5). A sample of OERs (n=27) rated by a small group (4) was conducted to determine inter-rater reliability. Intra-class correlation indicated moderate agreement (ICC(2,1) =.655, df=376, 95% CI [.609, .699]). Findings suggested that the degree of technical difficulty in reusing OERs can be measured in somewhat reliable manner. These findings may be insightful in developing policies and practices regarding OER development.



College and Department

David O. McKay School of Education; Instructional Psychology and Technology



Date Submitted


Document Type





OER, Open Educational Resources, ALMS, Delphi study