# README [![CircleCI](https://circleci.com/gh/Lumieducation/H5P-Nodejs-library/tree/master.svg?style=svg)](https://circleci.com/gh/Lumieducation/H5P-Nodejs-library/tree/master) [![Coverage Status](https://coveralls.io/repos/github/Lumieducation/H5P-Nodejs-library/badge.svg?branch=master)](https://coveralls.io/github/Lumieducation/H5P-Nodejs-library?branch=master) This library provides everything needed to create custom H5P servers running on NodeJS. It is written in TypeScript and fully typed, which makes it much easier to work with than the official PHP server. Of course, it's also possible to use this library in projects with JavaScript (ES5) and you will still profit from the typings by getting code completion in your IDE. Even though the repository includes a sample demo project that demonstrates its functionality, it's still your job to integrate this library into your own NodeJS server. This is called an "implementation" or "plugin" in H5P terminology. The implementation is responsible for exposing HTTP endpoints, persisting data and calling maintenance functions. **This library is not an out-of-the-box solution to get a standalone H5P server.** **Check out the** [**GitBook documentation**](https://docs.lumi.education) **for details on how to use this library**. Please note that even if most functionality of H5P works, **there are parts which haven't been implemented yet or which might be faulty.** This is particularly true for security concerns. For a more comprehensive list of what works and what doesn't, check out [the documentation page on the current status of the project](/development/status). The interfaces have reached some level of stability, but might still change in future major releases. ## Packages The main Git repository is a monorepo that contains several packages, which can be installed through NPM. The packages are meant to be combined. | Package name | Functionality | used in | | -------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- | -------- | | **@lumieducation/h5p-server** | the core package to run H5P in NodeJS | backend | | **@lumieducation/h5p-express** | routes and controllers for Express | backend | | [**@lumieducation/h5p-webcomponents**](/npm-packages/h5p-webcomponents) | native web components to display the H5P player and editor in the browser | frontend | | [**@lumieducation/h5p-react**](/npm-packages/h5p-react) | React components with the same functionality as the native web components | frontend | | [**@lumieducation/h5p-mongos3**](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/docs/packages/h5p-mongos3/README.md) | storage classes for MongoDB and S3 | backend | | [**@lumieducation/h5p-redis-lock**](/npm-packages/h5p-redis-lock) | storage class for locks with Redis | backend | | **@lumieducation/h5p-html-exporter** | an optional component that can create bundled HTML files for exporting | backend | | [**@lumieducation/h5p-svg-sanitizer**](/npm-packages/h5p-svg-sanitizer) | an optional package that protects against XSS attack in SVGs if you want to enable SVG in content files | backend | | [**@lumieducation/h5p-clamav-scanner**](/npm-packages/h5p-clamav-scanner) | an optional package that checks file uploads for viruses | backend | ## Examples There are two example implementations that illustrate how the packages can be used: | Example type | Tech stack | Location | | -------------------------------------- | ------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | | server-side-rendering | server: Express with JS template rendering client: static HTML, some React for library management | `/packages/h5p-examples` | | [Single Page Application](/usage/rest) | server: Express with REST endpoints client: React | `/packages/h5p-rest-example-server` `/packages/h5p-rest-example-client` | ## Trying out the demo Make sure you have [`git`](https://git-scm.com/), [`node ≥ 10.16`](https://nodejs.org/) , and [`npm`](https://www.npmjs.com/get-npm) installed. If you use Windows, you must use bash (comes with Git for windows) as a command shell (otherwise scripts won't run). 1. Clone the repository with git 2. `npm install` 3. `npm start` You can then open the URL in any browser. ## Contributing Lumi tries to improve education wherever it is possible by providing a software that connects teachers with their students. Every help is appreciated and welcome. Feel free to create pull requests. Check out the [documentation pages for developers](/development/getting-started) to get started. This project has adopted the code of conduct defined by the Contributor Covenant. It can be read in full [here](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/code-of-conduct.md). ## Get in touch [Slack](https://join.slack.com/t/lumi-education/shared_invite/zt-3dcc4gpy-8XxjefFeUHEv89hCMkwmbw) or [c@Lumi.education](mailto:c@lumi.education). ## Versioning We use [SemVer](http://semver.org/) for versioning. For the versions available, see the [tags on this repository](https://github.com/Lumieducation/Lumi/tags). ## License This project is licensed under the GNU GENERAL PUBLIC LICENSE v3 License - see the [LICENSE](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/LICENSE/README.md) file for details ## Support This work obtained financial support for development from the German BMBF-sponsored research project "lea.online" (FKN: W-143600). Read more about them at the following websites: * lea.online - * University of Bremen - * BMBF - # Basic usage To find out what this library provides for you and what you must implement on your own, check out the [architecture overview](/usage/architecture) first. Then, read about how to [integrate @lumieducation/h5p-server](/usage/integrating) into your own NodeJs application. Next, you must [add several HTTP routes to your application](/usage/ajax-endpoints) to serve AJAX requests that are needed by the H5P core to work. Finally, read about the [options you have when creating an H5PEditor object](/usage/h5p-editor-constructor). # Architecture ## General overview A H5P web-application using this library consists of four components, which communicate between each other: 1. This library (@lumieducation/h5p-server and possibly other @lumieducation packages) on the server-side (in yellow below) 2. Your server (implementation of the interfaces of this library + other endpoints, in blue below) 3. Your web client (running in the browser, in blue below, @lumieducation/h5p-webcomponents and @lumieducation/h5p-react can help your here if you want to write a SPA application) 4. Joubel's H5P player / editor client (downloaded by you and served by your server, in grey below) As you can see, this library is not an out-of-the-box solution for all your needs, but **requires you to implement your own server and web client**. ## Example diagram of the editor The diagram shows how the four components interact in a selection of use cases. Each use case has a specific colour and can be traced through the system by it. ![Diagram showing the components at work](/files/-MQcK_ikukhvzGQb8YyL) You have to implement all the components shown in blue. This library (@lumieducation/h5p-server) provides the parts in yellow and the grey parts are provided by Joubel's H5P client libraries (downloaded from the PHP implementation). The Express package (@lumieducation/h5p-express) deals with all AJAX calls by the H5P JavaScript client. You are free to use it, but can also implement the HTTP endpoints yourself. Note: The diagram doesn't show a complete list of all use cases and is only intended to illustrate how everything plays together! There is a lot more functionality in the packages, which is not listed here. The player is simpler, as it doesn't have as many endpoints, but works in a comparable way to the structure in the diagram. ## Example implementation There is a basic example implementation of the server and client (blue parts) using Express (and server-side rendering) in the `/packages/h5p-examples` folder. There is also a more advanced SPA example in the `/packages/h5p-rest-example-server` and `/packages/h5p-rest-example-client` folders. Check out the documentation on it [here](/usage/rest). # Integrating the core library ## Installation Add the library to a project by executing ```bash npm install @lumieducation/h5p-server ``` ## Adding the library to your project *Note: The example snippets below use the* [*ES2017 async await language features*](https://javascript.info/async-await) *to simplify dealing with promises. You can still use the traditional .then(()=> {...}) style if you wish so.* After installation, you can import the library in a JavaScript file with ```javascript const H5P = require('@lumieducation/h5p-server'); ``` and instantiate the editor with ```javascript const h5pEditor = H5P.fs( await new H5P.H5PConfig( new H5P.fsImplementations.JsonStorage( path.resolve('examples/config.json') // the path on the local disc // where the configuration file is stored ) ).load(), path.resolve('h5p/libraries'), // the path on the local disc where libraries // should be stored path.resolve('h5p/temporary-storage'), // the path on the local disc where // temporary files (uploads) should // be stored path.resolve('h5p/content') // the path on the local disc where content is // stored ); ``` This object is **a server-side component** that is long-lived and includes most of the methods that must be called when by the routes of your server. To find out what these routes are and how they must call H5PEditor, you can either check out the [section on content views in the docs](#creating-content-views) or the [Express example project](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-examples/src/expressRoutes.ts). To render a HTML page (server-side rendering) with the editor inside (= the HTML user interface around the actual editor and the editor itself) for a specific content ID, you call ```javascript const html = await h5pEditor.render(contentId, user); // send the html to the browser, which will display it ``` To use a custom renderer, change it with ```javascript h5pEditor.setRenderer(model => /** HTML string or object **/); ``` Check out the [default editor renderer](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-server/src/renderers/default.ts) for inspiration or customization possibilities. You can also use a custom renderer that returns a plain object with the data required to display an H5P editor instead of HTML markup. If you return this data to the browser as a response to an Ajax call, you can also create Single Page Applications that don't rely on server-side rendering. See [the documentation page on constructing a `H5PEditor` object](/usage/h5p-editor-constructor) for more details on how to instantiate the editor in a more customized way. You can create [`H5PPlayer`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-server/src/H5PPlayer.ts) objects in a similar way and use them in a similar fashion. ## Handling AJAX Requests made by the core H5P client The H5P client (running in the browser) sends many AJAX requests to the server (this application). While this library provides you with everything required to process the requests in the backend, your implementation must still serve the requests to these endpoints. There is an Express adapter that you can use out-of-the box for this purpose. Check out the [documentation on endpoints](/usage/ajax-endpoints) for details. ## Serving static H5P core files for the client This application doesn't include the H5P JavaScript core files for the editor and the player. These are the files that make up the editor and player that the end user interacts with in the browser. The core files must be obtained separately: 1. Download the [Core files](https://github.com/h5p/h5p-php-library/archive/1.24.0.zip) and place them into a folder called `h5p/core` in your project. 2. Download the [Editor files](https://github.com/h5p/h5p-editor-php-library/archive/1.24.1.zip) and place them into a folder called `h5p/editor` in your project. You must add a route to your implementation that serves the static files found under `h5p/core` and `h5p/editor` to the endpoint configured in `config.libraryUrl`. The out-of-the-box Express adapter already includes a route for this. ## Creating content views While the AJAX communication between the actual H5P editor client (running in the browser) and the server (this application) can be fully handled by the Express adapter, you must still create custom views for these purposes: * View that is shown when creating new content * View that is shown when editing existing content * View for deleting content * View that lists existing content * View that plays content The reason why you have to do this on your own is that this library is unaware of other data that your system might attach to a piece of content (e.g. access rights, tags). If you want any custom UI elements around the editor and player (which is highly likely), you must put this into the views. Check out the example for how to write custom views. ## Writing custom interface implementations Several aspects of your H5P server can be customized by creating your own implementation of interfaces and passing them to the constructor of `H5PEditor`. That way you can use a database of your choice, cache data in Redis or store user data in an object storage system. The interfaces that can be implemented are: * `IContentStorage` * `IH5PConfig` * `ILibraryStorage` * `ITemporaryFileStorage` * `IUser` There are already default implementations that you can use: * The implementations in the `fs` folder store all data in the local file system and are only for demonstration purposes and not suitable to be used in a multi-user environment and not optimized for speed. You might be able to use them in a cluster setup by using a network storage. * There is an implementation of the content storage for MongoDB and S3-compatible storage systems. Check out more information [in the documentation page](/npm-packages/h5p-mongos3/mongo-s3-content-storage). * There is an implementation of the temporary file storage for S3-compatible storage systems. Check out more information [in the documentation page](/npm-packages/h5p-mongos3/s3-temporary-file-storage). * There is an implementation of the library file storage for MongoDB and S3-compatible storage systems. ## Calling maintenance functions regularly The implementation needs to call several function regularly (comparable to a cronjob): * Call `H5PEditor.temporaryFileManager.cleanUp()` every 5 minutes. This checks which temporary files have expired and deletes them if necessary. It is important to do this, as temporary files are **not** automatically deleted when a piece of content is saved. * Call `H5PEditor.contentTypeCache.updateIfNecessary()` every 12 hours. This will download information about the available content types from the H5P Hub. If you don't do this, users won't be shown new content types or updates to existing content types when they become available. ## Handling errors If something goes wrong and a call to the library can't continue execution, it will normally throw either a `H5PError` or an `AggregateH5PError` (a collection of several errors). Both errors types represent errors that can be sent to the user to be displayed in the client (in the user's language). They don't include the English error message but an error id that you must translate yourself. Error ids and their English translations can be found in \[`/packages/h5p-server/assets/translations`]. The translation strings follow the format used by [i18next](https://i18next.com), but in theory you can use any localization library. Calls to the library might also throw regular `Error` objects. In this case the error is not caused by the business logic, but by some more basic functionality (file system, other library) or it might be an error that is addressed at the developer (i.e. because function parameters aren't correctly used). The Express adapter already catches errors, localizes them and returns proper HTTP status codes. Check out the implementation there for a guide how to deal with errors. ## Localization This library supports localization. See the [respective documentation page](/advanced-usage/localization) for more details. ## Customization An application using @lumieducation/h5p-server can customize the way H5P behaves in several ways. See [the documentation page on customization](/advanced-usage/customization) for more details. ## Compliance and privacy To conform with local law, you probably have to compile a privacy declaration for your application. You can check out the [documentation page on privacy](/advanced-usage/privacy) to find out what this library does with your users' personal data. # H5P Ajax Endpoints There are two ways of handling AJAX requests: You can use the out-of-the-box [Express adapter](#handling-requests-with-the-express-adapter) or [write your own custom router](#handling-requests-yourself): ![diagram of the architecture of the H5P Ajax endpoint](/files/-MU5klbE8B1ennUJ3Jy4) ## Handling requests with the Express adapter Your implementation must process requests to several endpoints and relay them to the H5PEditor or H5PPlayer objects. All Ajax endpoints are already implemented in the [Express adapter](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-express/src/H5PAjaxRouter/H5PAjaxExpressRouter.ts), which you can use like this: Import the Express adapter router like this: ```typescript import h5pAjaxExpressRouter from '@lumieducation/h5p-express'; ``` or in classic JS style: ```javascript const h5pAjaxExpressRouter = require('@lumieducation/h5p-express'); ``` Then add the router to your Express app like this ```javascript app.use( // server is an object initialized with express() '/h5p', // the route under which all the Ajax calls will be registered h5pAjaxExpressRouter( h5pEditor, // an H5P.H5PEditor object path.resolve('h5p/core'), // the path to the h5p core files (of the player) path.resolve('h5p/editor'), // the path to the h5p core files (of the editor) routeOptions, // the options are optional and can be left out languageOverride // (optional) can be used to override the language used by i18next http middleware ) ); ``` Note that the Express adapter does not include pages to create, editor, view, list or delete content! You can customize which endpoints you want to use by setting the respective flags in the `options` object. By default, the adapter will handle **all** routes and you can turn individual ones off by setting `routeXX` to false. You can also turn off the error handling (`handleErrors: false`). Normally, the router will send back localized responses that the H5P client can understand. If you turn error handling off, the routes will throw errors that you have to handle yourself! **IMPORTANT:** The adapter expects the requests object of Express to be extended like this: ```typescript { user: IUser, // must be populated with information about the user (mostly id and access rights) t: (errorId: string, replacements: {[key: string]: string }) => string } ``` The function `t` must return the string for the errorId translated into the user's or the content's language. Replacements are added to the localized string with curly braces: It is suggested you use [i18next](https://www.i18next.com/) for localization, but you can use any library, as long as you make sure the function t is added to the request object. ## Handling requests yourself If you use a different HTTP framework than Express, you can write your own adapter. In this case, you must instantiate `H5PAjaxEndpoint` and call its methods when your routes are called. The table below shows which routes you must implement and which ones can be left out. Note that routes of the Type *H5P* are needed by the H5P client and must be implemented in some way. Routes of the type *custom* are specific to @lumieducation/h5p-server. The exact name of the routes can be [configured in IH5PConfig](#configuring-custom-endpoint-routes) and might be different in your setup. | HTTP Verb | Route | method in H5PAjaxEndpoint | Type | Required | | --------- | ----------- | ------------------------- | ------ | ------------------------------------------------------------------------------------- | | GET | /ajax | getAjax | H5P | yes | | GET | /content | getContentFile | H5P | depends on content storage: files in FileContentStorage can also be served statically | | GET | /libraries | getLibraryFile | H5P | depends on library storage: files in FileLibraryStorage can also be served statically | | GET | /temp-files | getTemporaryFile | H5P | yes | | POST | /ajax | postAjax | H5P | yes | | GET | /params | getContentParameters | custom | if you use the default renderer script of the editor | | GET | /download | getDownload | custom | no | Consult the [documentation of `H5PAjaxEndpoint`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-server/src/H5PAjaxEndpoint.ts) for details on who to retrieve the required parameters from the HTTP requests. You can also look at the [Express Ajax Adapter](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-express/src/H5PAjaxRouter/H5PAjaxExpressController.ts) as an example. ## Configuring custom endpoint routes The H5P client (run in the browser by the user) can be configured to use custom AJAX request endpoints. These can be configured in the config object. The relevant settings (including defaults) are: ```javascript const config = { ajaxUrl: '/ajax?action=', // URL prefix for all AJAX requests baseUrl: '/h5p', // a prefix added to all Ajax URLs contentFilesUrl: '/content',// base path for content files (e.g. images, video) coreUrl: '/core', // URL of static player "core files" downloadUrl: '/download', // URL to download h5p packages editorLibraryUrl: '/editor',// URL of static editor "core files" (not the content types!) librariesUrl: '/libraries', // URL at which library files (= content types) can be retrieved paramsUrl: '/params' // URL at which the parameters (= content.json) of content can be retrieved playUrl: '/play' // URL at which content can be displayed ... // further configuration values } ``` # Constructing H5PEditor There are two ways of creating a H5PEditor object: * You can use the convenience function [`H5P.fs(...)`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-server/src/implementation/fs/index.ts) that uses basic file system implementations for all data storage services. You can use the function if you're just getting started. Later on, you'll want to construct the editor with custom implementations of the data storage services. Check out the JSDoc of the function for details how to use it. * You can construct it manually by calling `new H5P.H5PEditor(...)`. The constructor arguments are used to provide data storage services and settings. You can find the interfaces referenced in [`src/types.ts`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-server/src/types.ts). Explanation of the arguments of the constructor: ## cache The `cache` object is used by the `ContentTypeCache` to persist information about content types. It might be also used for other functionality in the future. It must be able to store arbitrary nested objects and must implement the interface `IKeyValueStorage`. If used in a multi-machine or multi-process setup, the cache must be a single point of truth and work across all processes. ## config An object holding all configuration parameters as properties. It must implement the `IH5PConfig` interface. You can use the sample implementation in [`/packages/h5p-server/src/implementation/H5PConfig.ts`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-server/src/implementation/H5PConfig.ts). ## libraryStorage The `libraryStorage` provides information about installed libraries and installs them. It must implement the `ILibraryStorage` interface. If you store all library information as files in folders under `./h5p/libraries` you can use the sample implementation in [`/packages/h5p-server/src/implementation/fs/FileLibraryStorage.ts`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-server/src/implementation/fs/FileLibraryStorage.ts): ```javascript const libraryStorage = new FileLibraryStorage(`h5p/libraries`); ``` ## contentStorage The `contentStorage` provides information about installed content and creates it. It must implement the `IContentStorage` interface. If you store all library information as files in folders under `./h5p/content` you can use the sample implementation in [`/packages/h5p-server/src/implementation/fs/FileContentStorage.ts`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-server/src/implementation/fs/FileContentStorage.ts): ```javascript const contentStorage = new FileContentStorage(`h5p/content`); ``` ## temporaryStorage When the user uploads files in the H5P editor client, these files are not directly stored alongside the content, either because the content hasn't been saved before (and there is no contentId) or because this might create left-over files if these files aren't used after all. Instead the server stores these files in a temporary storage system and adds the '#tmp' tag to the files' path. When the editor client requests a file the system retrieves the file from temporary storage instead of the regular content storage. The temporary storage must implement the interface `ITemporaryFileStorage`. Furthermore, you should regularly call `H5PEditor.temporaryFileManager.cleanUp()` to remove unneeded temporary files (every 5 min). If you don't have a multi-machine setup you can use the sample implementation in [`/packages/h5p-server/src/implementation/fs/DirectoryTemporaryFileStorage.ts`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-server/src/implementation/fs/DirectoryTemporaryFileStorage.ts): ```javascript const temporaryStorage = new DirectoryTemporaryFileStorage( path.resolve('h5p/temporary-storage') ); ``` The sample implementation has a basic authentication mechanism built in that makes sure that only users who have created a file can access it later. ## translationCallback (optional) If you want to localize certain aspects of the editor, you must pass in a function that returns translated strings for certain keys. This function in turn can call another translation library to perform the localization. We suggest using [i18next](https://www.npmjs.com/package/i18next) as the keys @lumieducation/h5p-server uses follow the conventions of i18next. You can still choose any translation library you like. ```typescript // Pass translationCallbackAdapter as a parameter to H5PEditor. const translationCallbackAdapter = (key, language) => { return i18NextTranslationFunction(key, { lng: language }); }; ``` The editor will fallback to English if you don't pass any translation callback. Also see the [documentation page on localization](/advanced-usage/localization) for more details. ## urlGenerator (optional) if you need to overwrite the logic to create the urls per request you can pass a custom url generator. It is possible to inherit from UrlGenerator and overwrite the `baseUrl` function if needed.The url generator can also be used to add CSRF tokens to POST URLs. See the third-party [H5PServer](https://github.com/BoBiene/H5PServer) project for an implementation sample using the urlGenerator. ## options (optional) Allows you to customize styles and scripts of the client. Also allows passing in a lock implementation (needed for multi-process or clustered setups). ## options.permissionSystem (optional) By passing in an implementation of `IPermissionSystem` you get fine-grained control over who can do what in your system. The library calls the methods of `IPermissionSystem` whenever a user performs an action that requires authorization. If you leave options.permissionSystem `undefined`, the library will allow everything to everyone! ## contentUserDataStorage (optional) The `contentUserDataStorage` handles saving and loading user states, so users can continue where they left off when they reload the page or come back later. It must implement the `IContentUserDataStorage` interface. # REST Example This repository illustrates how to use the the packages of [h5p-nodejs-library](https://github.com/lumieducation/h5p-nodejs-library) in a Single Page Application with a REST backend in TypeScript. Naturally the application is separated into a **client**, which uses React as a framework. The **server** implements the `@lumieducation/h5p-server` using Express. You are not bound to using React or Express in your own application, as `@lumieducation/h5p-server` is framework agnostic. Check out the architecture overview that describes which parts of the application are provided by which package: ![Architecture overview](/files/-MUJY8b1i0AolKpHxzDC) * This repository contains all the components in **purple** boxes. * The **green** parts come from one of the server-side packages of `h5p-nodejs-library`. While `@lumieducation/h5p-server` is also a dev dependency of the client, this is only the case to use the TypeScript interface definitions exported by it. The dependency (which can be rather large) is only required at build time and not at runtime. * The **red** parts are React components that can be found in the package `@lumieducation/h5p-react`. The React components wrap around the web components from `@lumieducation/h5p-webcomponents`, which in turn wrap the actual core h5p player and editor JavaScript and simplify instantiation, loading, saving and event handling. * The **blue** parts is comprised of JavaScript and CSS files that make up the core H5P player and editor. They are part of the original PHP repositories and are downloaded from GitHub in the server with the script [`download-core.sh`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-examples/download-core.sh). They must be served as static files by the server and are added to the page by the web components as needed. ## Trying it out 1. Clone the repository 2. Run `npm install` **in the root**. This will install all dependencies of the example packages and internally link the packages inside the monorepo. 3. Run `npm start` for the server. (Must be up and running before you start the client!) 4. Run `npm start` for the client (**the server must still be running**). 5. A browser should open on . If there is an error, you have to reload the the page, as the server might not be fully initialized yet. ## Client The client was bootstrapped with [Create React App](https://github.com/facebook/create-react-app). For more information see [its documentation page](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/docs/examples/rest/Client.md). ## Support This work obtained financial support for development from the German BMBF-sponsored research project "lea.online -" (FKN: 41200147). Read more about them at the following websites: * lea.online Blog (German) - blogs.uni-bremen.de/leaonline * University of Bremen - * BMBF - # Advanced usage # Authorization Many actions users perform in the H5P system need authorization. By default the library will allow everything to every user. You can customize who can do what, but passing in an implementation of `IPermissionSystem` into `options.permissionSystem` of the `H5PPlayer` or `H5PEditor` constructor. The library then calls the methods of `IPermissionSystem` whenever a user performs an action that requires authorization. See the documentation of `IPermissionSystem` for and the [`ExamplePermissionSystem`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-rest-example-server/src/ExamplePermissionSystem.ts) for reference how to implement the permission system. Note that the `IPermissionSystem` is a generic. You can use any sub-type of `IUser` as the generic type. The call of the methods of `IPermissionSystem` will include a user of the generic type. This is the user object you've injected in your controllers. That means you can add any arbitrary date to it, like roles. ## Acknowledgement The development of this feature was kindly funded by PHYWE Systeme GmbH und Co. KG (). # User content state The H5P client is capable of saving the current state of the user so that the user can resume where they left off. This means that e.g. their attempts entered into textboxes are the same as when they last left off. ## How it works * If state saving is enabled, a timer interval is set in the H5PIntegration object by the server * The H5P core client reads the interval and tells the content type that is currently being displayed to persist it's state into a JSON object * The H5P core client sends the state to an AJAX route on the server (specified in H5PIntegration). * The server stores the state in a special storage system. When content is deleted, the user state is deleted as well. When content is updated, the user state is deleted if the content type requests this (the case for (nearly?) all content types). * When the user later re-opens the content, the server checks if there is a user state for the user that should be "preloaded". This means that the initial information about the content object also includes the state and the client doesn't have to make a second request to get it. If the state is marked as "preloaded" (this is done by the content type), the content type uses it during it's initialization routine. * If "preloaded" is set to `false` the H5P client can also request the user state through an AJAX call from the server. * There can also be a user state in the editor. For instance, it saves whether the user has dismissed the tours of Interactive Video or has closed one of the yellow "information boxes" that explain functions of the editor. The editor always gets the state through a second AJAX call. ## Limitations of the user state * Not all content types implement it. * Not all content types fully restore the state (e.g. they don't restore if the user has already pressed "checked"). ## Enabling user state * Create an instance of `IContentUserDataStorage`. The recommended storage class for production is `MongoContentUserDataStorage` in the `@lumieducation/h5p-mongos3` package. There's also a `FileContentUserDataStorageClass` in the `@lumieducation/h5p-server` package that you can use for development or testing purposes. * Pass the implementation of IContentUserDataStorage into the `H5PEditor` and `H5PPlayer` constructor. * Set `contentUserStateSaveInterval` in `IH5PConfig` to the interval at which the client should save the state (in milliseconds). The recommended number is `10000`. (To disable the feature, set `contentUserStateSaveInterval` to `false`) * If you use `h5pAjaxExpressRouter` from the `@lumieducation/h5p-express` package, then the routes for the AJAX endpoint are automatically created. You can manually turn them on by setting `routeContentUserData` in the options when creating the route. * If you don't use `h5pAjaxExpressRouter`, you have to route everything manually. First get `ContentUserDataManager` from `H5PEditor` or `H5PPlayer`. Route these endpoints to the functions and return HTTP status code 200 with a JSON object that is based on `AjaxSuccessResponse` with empty payload (Check out the Express Router for details): * GET {{contentUserDataUrl}}/:contentId/:dataType/:subContentId -> `ContentUserDataManager.getContentUserData` * POST {{contentUserDataUrl}}/:contentId/:dataType/:subContentId -> `ContentUserDataManager.createOrUpdateContentUserData` ## Configuration options * You can customize the URL at which the AJAX calls are available by setting `contentUserDataUrl` in `IH5PConfig`. * You can customize the interval at which content states are saved by setting `contentUserStateSaveInterval` in IH5PConfig. If you set it to false, you can disable the feature. ## Security considerations You should implement CSRF tokens when using the content user state as the POST endpoint would otherwise by vulnerable to CSRF attacks when using cookie authentication. The tokens are added to the endpoint URL in the IUrlGenerator implementation and thus sent to the server whenever a POST call is made. Check out the REST example on how to pass the CSRF token to the H5P server components and how to check its validity. # Multiple user states per object You can save multiple user states (= the data a learner entered, e.g. the attempts in a fill-in-the-blanks activity) for one user - content tuple. This is a useful feature, if you want to allow your users to have more than one attempt per content object and if they can return to older ones. This feature is unique to this H5P NodeJs implementation and not part of the standard H5P PHP server version. ## Usage If you want to use this feature, your implementation has to provide a `contextId` in the `H5PPlayer.render` options: ```js const html = await h5pPlayer.render( contentId, // the content id user, // the current user 'auto', // automatic language detection { contextId: '' // you can add more options here as well } ); ``` Context ids can't be used in the H5P editor, as the user state in the editor is only used to save things like whether certain info boxes were collapsed. Multiple contexts wouldn't make sense there. ## Creating contextId `contextId` is an arbitrary string value that you define in your implementing system. It is **your** job to keep it unique and to pass it the the `render` method. Typically it is associated with some other object in your database anyway (e.g. an attempt object), so you will already have a unique id that you can use here. ## Using contextIds with old data It is save to use `contextIds` if you already have user content data in your system that didn't use contextIds. You can also have user data that has a `contextId` and other user data that doesn't in the same system. ## Web Components and React The H5P Player Web Component and React component also support the context id. You can set the current context id by setting the attribute/property `contextId` to the desired value. Make sure that you implementation of `loadContentCallback` accepts `contextId` as the second parameter, that you include it in the request to the server and that your server passes the contextId to `H5PPlayer.render`. ## Storage support Both the `FileContentUserDataStorage` and `MongoContentUserDataStorage` support context ids. ## Trying it out in the examples The server-side-rendering example supports context ids. You can set the context id by passing it as a query parameter in the URL, e.g. `http://localhost:8080/h5p/play/?contextId=` where `` is an arbitrary value you can make up on the fly. The REST example also supports context ids. You can see the currently used context id in the user interface after the `#` icon. You can change the current context id with the button. ## Acknowledgement The development of this feature was kindly funded by PHYWE Systeme GmbH und Co. KG (). # Impersonating users It is possible to impersonate users when viewing a H5P object. This means that you can display another user's user state instead of your own. This is useful, if you want to implement a feature in which teachers can review the work of students. You do this by setting `options.asUserId` of the `H5PPlayer.render` method. Make sure that you [authorize users](/advanced-usage/authorization) as required in the permission system. ## Read-only states In most cases in which your users impersonate another user, you'll want to disable saving the user state for the impersonator. You can do this by setting `options.readOnlyState` to true when calling `H5PPlayer.render`. This will do the following: * set the save interval to the longest possible value * adds the query parameter `ignorePost=yes` to the Ajax route responsible for handling user states The query parameter is necessary, as the H5P core client doesn't support user states that are read only. We work around this by ignoring a post calls when the query parameter is set in h5p-express. If the query parameter is set, we simply return a success, so the H5P core client doesn't realize we didn't save the state. If you don't use this package, you must do this yourself. Obviously you also have to make sure malicious users won't change the user state of others, by rejecting these operations in the authorization/permission system! ## Trying it out in the examples The server-side-rendering example supports impersonation and read-only state. You can use them by passing in query parameter in the URL, e.g. `http://localhost:8080/h5p/play/?asUserId=&readOnlyState=yes`. The REST example also supports impersonation and read-only states. You can enable these features in the web interface. ## Acknowledgement The development of this feature was kindly funded by PHYWE Systeme GmbH und Co. KG (). # Basic completion tracking The H5P client is capable of sending a message to the server when the user has completed a content object. This includes the time when this occured, how long the content was open, the achieved score and the maximum score. While technically this message is derived from a xAPI statement generated by the content types, it **is not the same as xAPI** and is a completely separate system that co-exists with xAPI and a potential LRS. You can enable completion tracking and xAPI tracking indepedently. If you want to capture all xAPI statements, which allows you to have very detailed tracking, you either have to inject your own xAPI capturing JavaScript or use the xAPI capabilities of the `H5PPlayerComponent` in the [webcomponent package](/npm-packages/h5p-webcomponents)) (or the corresponding functionality in the [React package](/npm-packages/h5p-react)). ## How it works * When the user presses "check", a xAPI statement indicating completion is generated by the content type (that supports it). The H5P client captures it and calls an AJAX route on the H5P server with basic information. * The H5P server saves the completion data in a special storage system. The server automatically deletes the data when the content object is deleted. ## Limitations * While the storage classes support retrieving and deleting the completion data, there are no endpoints on the `h5p-express` package that implement this functionality. You have to implement these endpoints yourself. * Not all content types emit xAPI statements indicating completion and thus the tracking isn't fired. ## Enabling completion tracking * Create an instance of `IContentUserDataStorage`. The recommended storage class for production is `MongoContentUserDataStorage` in the `@lumieducation/h5p-mongos3` package. There's also a `FileContentUserDataStorageClass` in the `@lumieducation/h5p-server` package that you can use for development or testing purposes. * Pass the implementation of IContentUserDataStorage into the `H5PEditor` and `H5PPlayer` constructor. * Set `setFinishedEnabled` in `IH5PConfig` to `true`. * If you use `h5pAjaxExpressRouter` from the `@lumieducation/h5p-express` package, then the routes for the AJAX endpoint are automatically created. You can manually turn them on by setting `routeFinishedData` in the options when creating the route. * If you don't use `h5pAjaxExpressRouter`, you have to route everything manually. First get `ContentUserDataManager` from `H5PEditor` or `H5PPlayer`. Route this endpoint and return HTTP status code 200 with a JSON object that is based on `AjaxSuccessResponse` with empty payload: * POST {{setFinishedUrl}}/ -> `ContentUserDataManager.setFinished` ## Configuration options * You can customize the URL to which the AJAX calls are made by setting `setFinishedUrl` in `IH5PConfig`. * You can enable or disable the feature by setting `setFinishedEnabled` in IH5PConfig. ## Security considerations You should implement CSRF tokens when using completion tracking as the POST endpoint would otherwise by vulnerable to CSRF attacks when using cookie authentication. The tokens are added to the endpoint URL in the IUrlGenerator implementation and thus sent to the server whenever a POST call is made. Check out the REST example on how to pass the CSRF token to the H5P server components and how to check its validity. # Localization ## Localizing the player The H5P player doesn't require localization as all the language strings are part of the H5P content package and must be set when creating a H5P package in the editor. ## Localizing the editor ### Places at which localization must happen To change the language of the H5P editor, the text strings must be localized in several places: 1. The core language strings (found in language/xxx.js) must be referenced in the HTML file and in the array which references the JS files in the `IIntegration` object (to make sure the language is also used in iframes). 2. The H5P editor client (running in the browser) must be notified to use a certain language. It will then request the respective localized strings of H5P libraries it loads. 3. Several string properties of IIntegration must be returned localized. 4. The errors thrown by @lumieducation/h5p-server must be localized. Some places of H5P cannot be localized at this time (this must be changed by Joubel): * Some strings in libraries that are hard-coded ### Changing the language of the editor @lumieducation/h5p-server supports localizing the editor as far as possible. The table shows where this must be done: | Place | What to do | | ------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 1. core language strings | Call `H5PEditor.render(contentId, language, ...)` with the language code you need. | | 2. notify H5P editor client | Call `H5PEditor.render(contentId, language, ...)` with the language code you need. | | 3. properties of IIntegration | Pass a valid `translationCallback` of type `ITranslationFunction` to the constructor of `H5PEditor` | | 4. error messages emitted by @lumieducation/h5p-server | Catch errors of types `H5PError` and `AggregateH5PError` and localize the message property yourself. | | 5. H5P Hub | When constructing `H5PEditor` set the option `enableHubLocalization` to true and load the namespace `hub` in your localization system. Call `H5PEditor.getContentTypeCache()` with a language or make sure that `req.language` is set in the GET AJAX route when using `h5p-express`. | | 6. library selector | When constructing `H5PEditor` set the option `enableLibraryNameLocalization` to true and load the namespace `library-metadata` in your localization system. Call `H5PEditor.getLibraryOverview()` with a language or make sure that `req.language` is set in the POST AJAX route when using `h5p-express`. | The [Express example](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-examples/src/express.ts) demonstrates how to do 1,2 and 3. The [Express adapter for the Ajax endpoints](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-express/src/H5PAjaxRouter/H5PAjaxExpressRouter.ts) already implements 4 but requires the `t(...)` function to be added to the `req` object. The language strings used by @lumieducation/h5p-server all follow the conventions of [i18next](https://www.npmjs.com/package/i18next) and it is a good library to perform the translation for cases 3 and 4. However, you are free to use whatever translation library you want as long as you make sure to pass a valid `translationCallback` to `H5PEditor` (case 3, 5 and 6) and add the required `t(...)` function to `req` (case 4). ### Initializing the JavaScript H5P client (in the browser) The H5P client must set the `H5PEditor.contentLanguage` property like this to localize the libraries when the editor is initialized : ```javascript H5PEditor.contentLanguage = H5PIntegration.editor.language; ``` You should include this initialization routine in your "create" and "edit" views. ### Language detection While you can manually change the language used in the Express adapter with the `languageOverride` parameter, it is best to use a language detector, which makes sure the req.t method uses the required target language. The easiest way is to implement your own language detector in i18next as explained in their [documentation](https://github.com/i18next/i18next-http-middleware#adding-own-detection-functionality). Initialize i18next with this detector (which can also simply return a hard-coded language if you want to only support one), and the Express adapter will translate to this language. ### Contributing to the translation strings H5P is constructed in a way that spreads out the localization effort. While the great majority of the language strings come packaged with the content types or are part of the H5P core (case 1 and 2 from the table), some strings must be localized by the server implementation. The Drupal, WordPress and Moodle PHP implementation all come with their own translation system and set of language strings. That's why @lumieducation/h5p-server must also follow this path and localize strings itself. The language strings used by @lumieducation/h5p-server can be found in `/packages/h5p-server/assets/translations/`. In there, each namespace (group of language strings) has it own directory, which in turn contains the language files, which are named like this `en.json`, `de.json` etc. If you want to change the text for your language or add another language, you must do the changes in these directories. You can also add new namespaces if you want to contribute to the development of @lumieducation/h5p-server and develop a module which is self-contained (like the optional storage implementations). All general language strings should be put into the namespace `server`. # Cluster The example can (mostly) be run in cluster mode to scale h5p horizontally. The only thing that hasn't been made to work across is the configuration object. (This shouldn't be a problem, as the only thing that is written back in the config is the UID of the server, which is already built in for development purposes.) All other components support scaling across several instances. You'll need: * Docker * Docker Compose You can access the application by calling from your browser. ## Commands Change directories to the directory that contains the `docker-compose.yml` file. (`packages/h5p-examples/cluster-mode`) ### Startup (fires up 4 instances of h5p) ```bash docker-compose up --scale h5p=4 ``` ### Startup (also rebuilds image) Execute this command after you've made changes to the code that require a rebuild ```bash docker-compose up --scale h5p=4 --build ``` ### Shutdown ```bash docker-compose down ``` ### Shutdown (and delete all data) ```bash docker-compose down -v ``` ## How it works The `docker-compose.yml` file configures a setup in which there are several services that make @lumieducation/h5p-server work in cluster mode: * Minio (provides S3 storage backend for content, temporary and library files) * MongoDB (provides database backend for content and library metadata) * Redis (provides a key-value cache; used for caching the content type cache and caching library metadata; used as a locking mechanism across containers) * a named Docker volume (added as a volume to all h5p containers to keep library data consistent across instances) * NGINX (load balancer that distributes incoming request between the H5P containers) # Addons Addons are H5P libraries that can be added to the H5P Editor or to certain H5P content without changing the content itself or the content type. They are a little-known feature of the H5P core and currently the only publicly known example of an addon is the [H5P.MathDisplay addon](https://h5p.org/mathematical-expressions), which uses MathJax to display mathematical formulas. Addons provide a simple and reusable way of customizing behavior and looks of H5P content types. This allows site administrators to customize H5P without programming knowledge or the need to fork content types. ## Using addons (for site administrators) To use an addon on your site, you must upload the addon file through the library management site. You **cannot** upload addons in the normal way you would upload a H5P package with content, as the .h5p files of addons don't contain any content and won't pass validation! Depending on the configuration of the addon and your server, the addon will now be automatically loaded in certain situations. As the only addon currently is H5P.MathDisplay, the procedure of getting addons to load will be explained with it below: **Player:** The MathDisplay addon will automatically be used in the player whenever needed, as the server scans content for a search string specified by the addon. You can also force the use of addons by setting the configuration property `playerAddons` of your configuration: ```javascript { // ... further configuration values ... "playerAddons": { "H5P.CoursePresentation": ["H5P.MathDisplay"] } // ... further configuration values ... } ``` **Editor:** The MathDisplay addon will **not** be automatically enabled in the editor. There are two ways to enable it: 1. Use a custom H5P.MathDisplay addon, that uses a lumieducation extension to avoid the server-wide configuration below. 2. Set the configuration property `editorAddons` in your implementation of `IH5PConfig` to something like: ```javascript { // ... further configuration values ... "editorAddons": { "H5P.CoursePresentation": [ "H5P.MathDisplay" ], "H5P.InteractiveVideo": [ "H5P.MathDisplay" ], "H5P.DragQuestion": [ "H5P.MathDisplay" ] } // ... further configuration values ... } ``` Now the editor will load the H5P.MathDisplay library if a user opens the editor of one these three content types (these are the three content types for which the PHP implementation also loads addons). ## Customizing addon behavior Addons can be configured by setting the property `libraryConfig` of your configuration implementation of `IH5PConfig`. The property is a complex object with H5P library machine names as keys. The object is sent to the H5P client (run in the browser) as part of the H5PIntegration object and can be accessed in a H5P library by calling `H5P.getLibraryConfig('H5P.MachineName')`. Example (shows how to (optionally) configure the [H5P.MathDisplay addon](https://h5p.org/mathematical-expressions)): ```javascript "libraryConfig": { "H5P.MathDisplay": { "observers": [ { "name": "mutationObserver", "params": { "cooldown": 500 } }, { "name": "domChangedListener" }, { "name": "interval", "params": { "time": 1000 } } ], "renderer": { "mathjax": { "src": "https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js", "config": { "extensions": ["tex2jax.js"], "jax": ["input/TeX", "output/HTML-CSS"], "tex2jax": { "ignoreClass": "ckeditor", "processEscapes": true }, "messageStyle": "none" } } } } } ``` ## Creating addons (for developers) Addons also use the .h5p file extension but don't have the same structure as regular h5p packages: they only contain folders with libraries and no `h5p.json` file or content. The library folders are basically like regular H5P libraries, as they contain a `library.json` file, but they can't contain `semantics.json`. The metadata in `library.json` is mostly the same as the metadata of normal libraries, but it contains the property `addTo`, which makes a library to an addon. (Check out the comprehensive structure of library metadata in the TypeScript interface `ILibraryMetadata` in [`/src/types.ts`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-server/src/types.ts).) A library containing the property `addTo` in its metadata will be automatically added to the player (or editor) by the server in certain circumstances: * Always when loading the editor, if set in the global configuration of the server (see above). * When playing content, only if the content contains a regex search string. The search string is set by setting this property: ```javascript { // ... more metadata ... "addTo": { "content": { "types": [ { "text": { "regex": "/your regex string/" // the regex string must start and end with a slash! } } ] } } // ... more metadata ... } ``` The configuration above means that the addon is added to every player instance if the regex is matched in any `string` property of the content parameters. * Always when loading the editor if requested in the metadata like this: ```javascript { // ... more metadata ... "addTo": { "editor": [ "H5P.CoursePresentation", "H5P.InteractiveVideo" ] } // ... more metadata ... } ``` *Note that this way of enabling addons in the editor is a custom lumieducation extension of the library metadata structure and is **not** supported by the PHP implementation and might change in the future if this feature is implemented by Joubel's PHP implementation in another way.* * Always when loading the play if requested in the metadata like this: ```javascript { // ... more metadata ... "addTo": { "player": [ "H5P.CoursePresentation", "H5P.InteractiveVideo" ] } // ... more metadata ... } ``` *Note that this way of enabling addons in the player is a custom lumieducation extension of the library metadata structure and is **not** supported by the PHP implementation and might change in the future if this feature is implemented by Joubel's PHP implementation in another way.* When an addon is loaded in the editor or the player, the JavaScript and CSS files listed at `preloadedJs` and `preloadedCss` are loaded in the HTML file after the actual library is loaded. # Customization An application using @lumieducation/h5p-server can customize the way H5P behaves in several ways: * You can **add global custom JavaScript or CSS files to the player and editor by passing their URL to the constructor of `H5PPlayer` or `H5PEditor`.** Use this method if you want to globally customize how H5P looks or behaves to make it fit into your larger application. [See below](#adding-custom-scripts-and-styles-via-constructor-injection) for more details. * You can **add global custom JavaScript or CSS files to the player and editor by specifying them in the configuration**. Use and document this method if you want to allow administrators of an instance of your application to customize how H5P looks and feels in their instance. [See below](#adding-custom-scripts-and-styles-via-the-configuration) for more details. * You can upload addons. This makes changing how H5P looks and behaves particularly easy, but requires the creation of addons. See the [addon documentation page](/advanced-usage/addons) for details. * You can **alter the scripts and styles used in a single library**. This allows you to change the looks and behavior of a library without forking it. [See below](#changing-javascript-and-css-files-of-individual-libraries) for more details * You can **alter the semantics and language files of a single library**. This allows you to change the default editor of libraries without forking the library itself. [See below](#changing-the-semantics-of-individual-libraries) for more details. To get a conceptual idea of how customizing works, you can also look at the official H5P documentation pages on ["Authoring tool customization"](https://h5p.org/documentation/for-developers/authoring-tool-customization) and ["Change the color of the editor"](https://h5p.org/change-color-of-the-editor). Of course, all the programming guides there only apply to the PHP implementation and not this to NodeJs version, but the basic idea is the same! ## Adding custom scripts and styles via constructor injection You can add custom scripts and styles to the player by passing their URLs to the options parameter of the constructor of `H5PPlayer` or `H5PEditor`: ```typescript const player = new H5PPlayer( libraryStorage, contentStorage, config, integrationObjectDefaults, // set to undefined if unneeded urlGenerator, // set to undefined if unneeded translationFunction, { customization: { global: { scripts: [ '/url/of/script/1.js', 'https://example.com/external/script.js' ], styles: [ '/url/of/style/1.js', 'https://example.com/external/style.css' ] } } } ); ``` -or- ```typescript const editor = new H5PEditor( cache, config, libraryStorage, contentStorage, translationCallback, // set to undefined if unneeded urlGenerator, // set to undefined if unneeded { customization: { global: { scripts: [ '/url/of/script/1.js', 'https://example.com/external/script.js' ], styles: [ '/url/of/style/1.js', 'https://example.com/external/style.css' ] } } } ); ``` These scripts or styles will then be added to the end of the list of H5P scripts that are loaded when the player or editor are loaded. Note that the URLs in the arrays are simply appended to the list of core scripts and styles without any changes. This means that they are not passed into URL generator! ## Adding custom scripts and styles via the configuration You can add lists of global JavaScript or CSS files to the configuration like this: ```javascript { // ... further configuration values ... "customization": { "global": { "editor": { "scripts": [ "/url/of/script/1.js", "https://example.com/external/script.js" ], "styles": [ "/url/of/style/1.css", "https://example.com/external/style.css" ] }, "player": { "scripts": [ "/url/of/script/1.js", "https://example.com/external/script.js" ], "styles": [ "/url/of/style/1.css", "https://example.com/external/style.css" ] } } } // ... further configuration values ... } ``` These scripts or styles will then be added to the end of the list of H5P scripts that are loaded when the player or editor are loaded. Note that the URLs in the arrays are simply appended to the list of core scripts and styles without any changes. This means that they are not passed into URL generator! ## Changing JavaScript and CSS files of individual libraries You can change the list of JavaScript and CSS files used by individual libraries to modify their looks and behavior. It is possible to add or to remove files from the lists. To do this you must pass a hook to the options object of the `H5PEditor` or `H5PPlayer` constructor. The hook looks like this: ```typescript /** * This hook is called when the player creates the list of files that * are loaded when playing content with the library. * Note: This function should be immutable, so it shouldn't change the * scripts and styles parameters but create new arrays! * @param library the library that is currently being loaded * @param scripts the original list of scripts that will be loaded * @param styles the original list of styles that will be loaded * @returns the altered lists of scripts and styles */ const alterLibraryFilesHook = ( library: ILibraryName, scripts: string[], styles: string[] ): { scripts: string[]; styles: string[] } => { // We only alter the files of a single library if (library.machineName === 'H5P.Example') { return { // The function should be immutable, so we re-create the arrays scripts: [...scripts, '/url/of/script.js'], styles: [...styles, '/url/of/style.css'] }; } // We return the original list for all other libraries. return { scripts, styles }; }; ``` It is passed to `H5PPlayer` like this: ```typescript const player = new H5PPlayer( libraryStorage, contentStorage, config, integrationObjectDefaults, // set to undefined if unneeded translationFunction, urlGenerator, // set to undefined if unneeded { customization: { alterLibraryFiles: alterLibraryFilesHook } } ); ``` It is passed to `H5PEditor` like this: ```typescript const editor = new H5PEditor( cache, config, libraryStorage, contentStorage, translationCallback, // set to undefined if unneeded urlGenerator, // set to undefined if unneeded { customization: { alterLibraryFiles: alterLibraryFilesHook } } ); ``` ## Changing the semantics of individual libraries It is possible to change the semantic structure of individual libraries without uploading a fork of the library. This allows you to remove or add fields in the H5P Editor. You can also change what HTML tags are allowed in the CKEditor, for instance. As the structure of the language files (which include all the translations) is identical to the semantic structure of the library, you must also make sure that the language files are changed as well. @lumieducation/h5p-server allows you to pass two hooks in the options of the `H5PEditor` constructor to achieve this: ```typescript /** * This hook is called when the editor retrieves the semantics of a * library. * Note: This function should be immutable, so it shouldn't change the * semantics parameter but return a clone! * @param library the library that is currently being loaded * @param semantics the original semantic structure * @returns the changed semantic structure */ const alterLibrarySemanticsHook = ( library: ILibraryName, semantics: ISemanticsEntry[] ): ISemanticsEntry[] => { // We only change the semantic structure of one library if (library.machineName === 'H5P.TrueFalse') { // We create a new array, as the function is supposed to be immutable. return [ { name: 'title2', type: 'text', widget: 'html', label: 'Title 2', enterMode: 'p', tags: ['strong', 'em', 'sub', 'sup', 'h2', 'h3', 'pre', 'code'] }, ...semantics ]; } // We return an unchanged structure for all other libraries return semantics; }; /** * This hook is called when the editor retrieves the language file of a * library in a specific language. * Note: This function should be immutable, so it shouldn't change the * languageFile parameter but return a clone! * @param library the library that is currently being loaded * @param languageFile the original language file * @param language the language for which the entries should be changed * @returns the changed language file */ const alterLibraryLanguageFileHook = ( library: ILibraryName, languageFile: ILanguageFileEntry[], language: string ): ILanguageFileEntry[] => { // We only change the language file of one library if (library.machineName === 'H5P.TrueFalse') { // We create a new array, as the function is supposed to be immutable. return [ // The language file has the same structure as the semantics file // but only includes localizable fields. { name: 'title2', label: 'Title 2' }, ...languageFile ]; } // We return an unchanged language file for all other libraries return languageFile; }; ``` Add these hooks to the constructor of `H5PEditor` like this: ```typescript const editor = new H5PEditor( cache, config, libraryStorage, contentStorage, translationCallback, // set to undefined if unneeded urlGenerator, // set to undefined if unneeded { customization: { alterLibraryLanguageFile: alterLibraryLanguageFileHook, alterLibrarySemantics: alterLibrarySemanticsHook } } ); ``` The changes to the semantic structure apply system-wide to all instance of the library in the editor. It doesn't make sense to customize the player this way, as the semantic structure and language files are only used by the editor. Exporting (downloading) and uploading content in the same application (or an application that uses the same hooks) works, but only the content.json file follows the altered semantic structure. The actual library's semantic.json file is left unchanged in the exported package. This means that if you upload an exported package with an altered semantic structure of the content to a different site, you might bump into validation errors or unexpected behavior. # Performance optimizations There are a few ways in which the performance of `@lumieducation/h5p-server` can be improved: * [Caching library storage](#caching-library-storage) * [Serving the library files from a different system](#serving-the-library-files-from-a-different-system) * [Horizontal scaling](#horizontal-scaling) * [Database-based content storage](#database-based-content-storage) ## Caching library storage The [`CachedLibraryStorage`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-server/src/implementation/cache/CachedLibraryStorage.ts) class can be used to cache the most common calls to the library storage. This will improve the overall performance of the library quite a bit, as many functions need library metadata, semantics or language files, which they get from the library storage. When used for complex content types like Course Presentation, the `GET /ajax?action=libraries` endpoint becomes around 40x faster if you use the cached library storage! The class uses [the NPM package `cache-manager`](https://www.npmjs.com/package/cache-manager) to abstract the caching, so you can pass in any of the store engines supported by it (e.g. redis, mongodb, fs, memcached). See the documentation page of `cache-manager` for more details. This is how you use the storage: ```javascript import * as H5P from '@lumieducation/h5p-server'; // const H5P = require('@lumieducation/h5p-server'); // old-style require alternative const cachedStorage = new H5P.cacheImplementations.CachedLibraryStorage( new H5P.fsImplementations.FileLibraryStorage(localLibraryPath) // you can also pass in other implementation of ILibraryStorage ); ``` Check out how to construct the H5PEditor with the storage [here](/usage/h5p-editor-constructor). ## Serving the library files from a different system While you can use the inbuilt methods of `@lumieducation/h5p-server` to serve the library files (JavaScript and CSS files used by the actual content types) to the browser of the user, it is also possible to serve them directly, as they are simple static files. If you construct [`FileLibraryStorage`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-server/src/implementation/fs/FileLibraryStorage.ts) with `new FileLibraryStorage('/directory/in/filesystem')` all libraries will be stored in `/directory/in/filesystem`. You can simply serve this directory under the URL specified in `IH5PConfig.librariesUrl` (defaults to `/libraries`). If you put this directory into a NFS storage or a shared volume, you can even use different machines to serve the library files (vertical scaling)! You must make sure that patches to libraries that have changed the library files aren't lost due to caching. ## Horizontal scaling It is possible to use `@lumieducation/h5p-server` in a setup in which multiple instances of it are executed in parallel (on a single machine to make use of multi-core processors or on multiple machines). When doing this, pay attention to this: * If you use [caching](#caching-library-storage), you have to use a cache like Redis or memcached. Cache invalidation across multiple instances will not work with the simple default in-memory cache. * If you run multiple instances on a single machine, the library storage directory can be on the local filesystem and all instances can share it. * If you run multiple instances on multiple machines, the library storage directory must be put into a network share (NFS) or you must use shared volumes (Docker). * It is advised to use the Mongo/S3 content storage classes. [See below](#database-based-content-storage) for details. ## Database-based content storage The simple [`FileContentStorage`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-server/src/implementation/fs/FileContentStorage.ts) and [`DirectoryTemporaryFileStorage`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-server/src/implementation/fs/DirectoryTemporaryFileStorage.ts) are not suitable for production use, as they suffer from serious scaling issues when listing content objects. They should only be used for development and testing purposes or in very small deployments. It is advised to use the [MongoS3ContentStorage](/npm-packages/h5p-mongos3/mongo-s3-content-storage) and [S3TemporaryFileStorage](/npm-packages/h5p-mongos3/s3-temporary-file-storage) classes of the h5p-mongos3 package. You can also write your own custom content storage and temporary file storage classes for other databases. # Privacy You can use the following information to make sure you comply with your local regulations (e.g. GDPR). You can base your privacy declaration on parts of it. **Note that this list was compiled to the best knowledge of the contributor, but that the contributor cannot provide a legal guarantee that the information is correct. In particular there might have been changes to the code after this list was created and these changes might not be reflected in this document. In doubt you should consult the source code yourself and make sure the information provided here is correct!** *Last update of this document: 28th June 2020* ## Stored personal data The library itself does not store personal data as such. For example it keeps no user directory. ## Processed personal data The implementation passes some personal data to @lumieducation/h5p-server for H5P to work properly. This data is used in several places by the system and might also be visible to the user (and in the future: other users). * the user's id * the real name of the user (full name) * the language used by user * rights and permissions of the user (in general and relating to content) ## Usage-based personal data When the user does certain actions in the system, the following data is stored (if the user chooses to do so): * h5p content objects * includes all personal data entered by the user as metadata (e.g. revision history, description, etc.) * includes user id of creator * includes creation date and time * (if applicable) includes media contained in it * uploaded resources (images, videos etc.) * includes user id of the user who uploaded the file * includes upload date and time ### Retention times * h5p content objects: kept until explicitly deleted by the implementation * uploaded resources: * kept until corresponding h5p content object is deleted * temporary files (used in the editor) are automatically deleted after the time set in `temporaryFileLifetime` in IH5PConfig (defaults to 120 min); **If you use the S3 storage backend, you have to configure automatic deletion of temporary files yourself!** ### Logging At the moment, the library does not include a component that logs user actions at the domain level. However, if you start the library in debug mode, it emits a log which might contain personal information (all of the data mentioned in the sections above). It is not recommended to turn on debugging in a production environment that processes real user data. ### Communication with 3rd party Internet servers The @lumieducation/h5p-server **package** (on the server) communicates with the official H5P Hub at h5p.org to retrieve the current list of available content types and to download new or updated content types from the H5P Hub. When first contacting the H5P Hub, the @lumieducation/h5p-server requests the creation of a unique id **for the server**, which is then transmitted to the H5P Hub on every content type list update. It also transmits the following information about your server: * the IP address of the server * the supported core API version of H5P * the version of the PHP implementation the server imitates * a local id of the server (a hash value of the path of the server files) * that the server runs on @lumieducation/h5p-server and with which version * whether the site is a localhost, in a private network or accessible in the Internet (configured by you in the IH5PConfig) When using the H5P Editor, **the browser** requests files from several 3rd party servers. This means the user's IP address and other data included in a HTTP request is transmitted to these servers. * fonts.googleapis.com (to download fonts) * fonts.gstatic.com (to download fonts) * h5p.org (to download image resources for the hub) Due to the open and modular nature of H5P it is impossible to fully list what every content type does. You should be aware of the fact that some H5P content types access resources from 3rd party servers in the browser through AJAX calls or direct references, which means the IP address and other metadata is transmitted to these server. Content types like 'speak the words' transmit personal data (the user's voice) to 3rd party server (Google in this case, if the user's browser is Google Chrome). **It is your responsibility to check and document what individual content types you choose to install on your server do with your users' data!** # Forward proxy support @lumieducation/h5p-server has to make outgoing HTTPS requests to contact the H5P Hub. If your network requires the use of a forward proxy to reach the Internet, you must configure @lumieducation/h5p-server to use it for the HTTPS requests. There are two ways of enabling the proxy: 1. Set the `proxy` property in IH5PConfig like this: ```json { "proxy": { "host": "10.1.2.3", "port": 8080, "protocol": "https" // can also be left out or set to "http" if your proxy can't be accessed with https } } ``` 2. Set the HTTPS\_PROXY environment variable like this: `HTTPS_PROXY=http://10.1.2.3:8080` or `HTTPS_PROXY=https://10.1.2.3:8080` (depending on whether your proxy can be accessed with https). # Security ## Restricting library installation The concept of H5P that H5P packages don't just include the author's content, but all the required libraries that are needed to display the content creates some inherent risks. As the libraries in a H5P package include JavaScript, CSS and SVG files, which are run in the user's browser, H5P basically deliberately allows Cross-Site-Scripting to some users. You must make sure that only trustworthy administrators can install and update H5P content types and libraries by setting up a restrictive [permission system](/advanced-usage/authorization)! In addition, you must make sure that all H5P packages that are uploaded by administrators only contain safe and trustworthy H5P libraries! Never upload H5P packages that you got from third-party sites (without thoroughly checking their contents). It's best to get them directly from the H5P Hub - or if your needed content types aren't available there directly from the developer. (Note that it can also mess up your H5P libraries, if user's upload forks of H5P content types that use the same library names as the original content type. In this case you can't update the original content type any more, as there are already "newer" libraries (the fork) in your installation. This error is difficult to track down. Most users don't know that H5P packages contain the libraries and that they might inadvertently "infect" an installation by uploading content packages. As an application developer and site administrator you must make sure that this doesn't happen!) ## File sanitization ### What do file sanitizers do? File sanitizers remove unsafe content from files that users upload into a H5P system. A file sanitizer takes a temporary file in the file system of the app server and changes its contents so that it's safe. For example, it can remove inline JavaScript code from SVG files. It is normal behavior for a sanitizer to alter an uploaded file, because it might, for instance, parse the file contents into a tree, transform that tree and then re-serialize the data structure. In comparison, malware scanners (see next section), only check for malicious contents and don't alter files. File sanitizers are used for these user uploads: * uploaded individual (media) files inside the H5P editor (e.g. images) (calls to `H5PEditor.saveContentFile`) * the contents (= media files in `content` directory) of uploaded H5P packages (calls to `ContentStorage.copyFromDirectoryToTemporary` or `H5PEditor.uploadPackage`) * the contents (= media files in `content` directory) of H5P packages directly stored with `ContentStorage.saveContentFile` (deprecated method!) Library files are not sanitized, as library files always contain JavaScript. This means there's no need to filter them, as libraries are a potential security risk by design. You must make sure to restrict library installation and only use libraries from trusted sources to counter this risk! ### How to use file sanitizers You can add file sanitizers to your H5P Editor setup, by initializing the H5P Editor like this: ```ts const h5pEditor = new H5PEditor( // ... regular configuration ... // Add the sanitizers to the options parameter { fileSanitizers: [ sanitizer1, sanitizer2 ] } ); ``` The sanitizers must implement this interface: ```ts interface IFileSanitizer { /** The name of the scanner, e.g. SVG Sanitizer. Used in debug output */ readonly name: string; /** Sanitizes files. The original file is expected to be replaced by the * sanitized file, so there is no new path to the sanitized file.*/ sanitize(file: string): Promise; } enum FileSanitizerResult { Sanitized, NotSanitized, Ignored } ``` Note: Sanitization only works if you pass uploaded content files to `H5PEditor.saveContentFile` as temporary files, not as in-memory streams! ### Existing file sanitizers There's an SVG sanitizer that removes unsafe parts of SVGs in the [`@lumieducation/h5p-svg-sanitizer` package](/npm-packages/h5p-svg-sanitizer). The examples in `packages/h5p-examples` and `packages/h5p-rest-example-server` already use this sanitizer. You can write you own implementations of the interface to sanitize other file types. ### Using multiple sanitizers It is possible to use more than one sanitizer. The sanitizers are called in sequence, meaning that the sanitizers are called one after the other. The most typical use case for this is to have sanitizers for different file types. ### Other uses of the interface It is possible to use the sanitizer for other use cases, like reducing the resolution of images to reduce their file size. ## Malware scanning ### What does malware scanning do? Malware scanners check uploaded user files for malicious contents ("viruses", scams etc.). They return a scan result and the H5P core library immediately removes uploaded files from temporary storage and returns an error to the user if the scan was not positive. Malware scanners are used for these user uploads: * uploaded individual (media) files inside the H5P editor (e.g. images) (calls to `H5PEditor.saveContentFile`) * the contents (= media files in `content` directory) of uploaded H5P packages (calls to `ContentStorage.copyFromDirectoryToTemporary` or `H5PEditor.uploadPackage`) * the contents (= media files in `content` directory) of H5P packages directly stored with `ContentStorage.saveContentFile` (deprecated method!) Library files are not scanned, as library files always contain JavaScript. This means there's no need to filter them, as libraries are a potential security risk by design. You must make sure to restrict library installation and only use libraries from trusted sources to counter this risk! ### How to use malware scanners You can add malware scanners to your H5P Editor setup, by initializing the H5P Editor like this: ```ts const h5pEditor = new H5PEditor( // ... regular configuration ... // Add the sanitizers to the options parameter { malwareScanners: [ scanner1, scanner2 ] } ); ``` The sanitizers must implement this interface: ```ts interface IFileMalwareScanner { /** The name of the scanner, e.g. ClamAV */ readonly name: string; /** Scans a file for malware and returns whether it contains malware. */ scan( file: string ): Promise<{ result: MalwareScanResult; viruses?: string }>; } enum MalwareScanResult { MalwareFound, Clean, NotScanned } ``` Note: Malware scanning only works if you pass uploaded content files to `H5PEditor.saveContentFile` as temporary files, not as in-memory streams! ### Existing malware scanners There's a example scanner using ClamAV in the [`@lumieducation/h5p-clamav-scanner` package](/npm-packages/h5p-clamav-scanner). The examples in `packages/h5p-examples` and `packages/h5p-rest-example-server` can be configured to use it (see the package's docs for how to use it). You can write you own implementations of the interface to use any other malware scanner. ### Testing whether the malware scanners if correctly set up There are test files in this repo that you can use to check whether your virus scanner is set up correctly. These test files utilize [EICAR test files](https://www.eicar.org/download-anti-malware-testfile/), which contain a totally harmless special character sequency detected by virus scanner. * `test/data/validator/h5p-with-virus.h5p`(/test/data/validator/h5p-with-virus.h5p): contains an [EICAR test file](https://www.eicar.org/download-anti-malware-testfile/) (as an image); upload this H5P package through the package upload functionality; you should see an error message explaining that the malware scanner has found something * `packages/h5p-clamav-scanner/test/eicar.png`(/packages/h5p-clamav-scanner/test/eicar.png): is an [EICAR test file](https://www.eicar.org/download-anti-malware-testfile/); upload this image as a media file anywhere in the H5P editor; you should see an error message explaining that the malware scanner has found something ## CSRF The H5P core library performs XHR calls with a session based on cookies, which are vulnerable to cross site request forgery attacks, if you don't use CSRF protection. ### CSRF tokens You can add CSRF tokens to the URLs of XHR calls by enabling it in the `UrlGenerator`'s constructor: ```ts const urlGenerator = new UrlGenerator( h5pConfig, { protectAjax: true, protectContentUserData: true, protectSetFinished: true, queryParamGenerator: (user: IUser) => ({ name: 'query_parameter_name'; value: 'a_generated_csrf_token'; }) } ); ``` All URLs of vulnerable endpoints will then contain the token, e.g. `https://example.org/h5p/some_api_call?query_parameter_name=a_generated_csrf_token`. While using tokens in URLs is not ideal, as the tokens will be leaked in logs, proxies (if you don't use TLS) and potentially Referrer Headers, there is no alternative as the H5P core doesn't support using headers. You can must then use a CSRF protection middleware to check for the token and reject calls that don't contain it. The REST example contains a demonstration how to wire everything up when using CSRF tokens. ### More modern CSRF protection You should consider using Cookie settings like `SameSite=Strict` and/or restrict API access to trusted origins with CORS. # NPM packages # h5p-mongos3 # Mongo/S3 Content Storage There is an implementation of the `IContentStorage` interface that uses MongoDB to store the parameters and metadata of content objects and a S3-compatible storage system to store files (images, video, audio etc.). You can find it at [/packages/h5p-mongos3/src/MongoS3ContentStorage.ts](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-mongos3/src/MongoS3ContentStorage.ts). **Note:** You must create the S3 bucket manually before it can be used by `MongoS3ContentStorage`! ## Dependencies The implementation depends on these npm packages: * aws-sdk * mongodb **You must add them manually to your application using `npm install aws-sdk mongodb`!** ## Usage You must import the storage implementation via a submodule: ```typescript import { MongoS3ContentStorage, initS3, initMongo } from '@lumieducation/h5p-mongos3'; ``` or in classic JS style: ```javascript const { MongoS3ContentStorage, initS3, initMongo } = require('@lumieduation/h5p-mongos3'); ``` Initialize the storage implementation like this: ```typescript const storage = new MongoS3ContentStorage( initS3({ credentials: { accessKeyId: 's3accesskey', // optional if env. variable is set secretAccessKey: 's3accesssecret' // optional if env. variable is set }, endpoint: 'http://127.0.0.1:9000', // optional if env. variable is set region: 'us-east-1', // optional if env. variable is set, forcePathStyle: true }), ( await initMongo( 'mongodb://127.0.0.1:27017', // optional if env. variable is set 'testdb1', // optional if env. variable is set 'root', // optional if env. variable is set 'h5pnodejs' // optional if env. variable is set ) ).collection('h5p'), { s3Bucket: 'h5pcontentbucket' } ); ``` ### Notes * The function [`initS3`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-mongos3/src/initS3.ts) creates an S3 client using the `aws-sdk` npm package. * The function [`initMongo`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-mongos3/src/initMongo.ts) creates a MongoDB client using the `mongodb` npm package. * You can pass credentials and other configuration values to `initS3` and `initMongo` through the function parameters. Alternatively you can use these environment variables instead of using the function parameters: * AWS\_ACCESS\_KEY\_ID * AWS\_SECRET\_ACCESS\_KEY * AWS\_S3\_ENDPOINT * AWS\_REGION * MONGODB\_URL * MONGODB\_DB * MONGODB\_USER * MONGODB\_PASSWORD * You can change the MongoDB collection name `h5p` to any name you want. If the collection doesn't exist yet, it will be automatically created. * You can change the bucket `h5pcontentbucket` to any name you want, but you must specify one. You must create the bucket manually before you can use it. * The configuration object passed into `initS3` is passed on to `aws-sdk`, so you can set any custom configuration values you want. * To achieve greater configurability, you can decide not to use `initS3` or `initMongo` and instantiate the required clients yourself. * While Amazon S3 supports keys with up to 1024 characters, some other S3 systems such as Minio might only support less in certain situations. To cater for these system you can set the option `maxKeyLength` to the value you need. It defaults to 1024. ## Using MongoS3ContentStorage in the example The [example Express application](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-examples/src/express.ts) can be configured to use the MongoDB/S3 storage by setting the environment variables from above and these additional variables: * CONTENTSTORAGE=mongos3 * CONTENT\_MONGO\_COLLECTION * CONTENT\_AWS\_S3\_BUCKET An example call would be: ```bash CONTENTSTORAGE=mongos3 AWS_ACCESS_KEY_ID=minioaccesskey AWS_SECRET_ACCESS_KEY=miniosecret AWS_S3_ENDPOINT="http://127.0.0.1:9000" MONGODB_URL="mongodb://127.0.0.1:27017" MONGODB_DB=testdb1 MONGODB_USER=root MONGODB_PASSWORD=h5pnodejs CONTENT_AWS_S3_BUCKET=testbucket1 CONTENT_MONGO_COLLECTION=h5p npm start ``` ## Customizing permissions By default the storage implementation allows all users read and write access to all data! It is very likely that this is not something you want! You can add a function to the options object of the constructor of `MongoS3ContentStorage` to customize access restrictions: ```typescript getPermissions = ( contentId: ContentId, user: IUser ) => Promise; ``` The function receives the contentId of the object that is being accessed and the user who is trying to access it. It must return a list of permissions the user has on this object. Your implementation of this function will probably be an adapter that hooks into your rights and permission system. ## Increasing scalability by getting content files directly from S3 In the default setup all resources used by H5P content in the **player** (images, video, ...) will be requested from the H5P server. The H5P server in turn will request the resources from S3 and relay the results. This means that in a high load scenario, there will be a lot of load on the H5P server to serve these static files. You can improve scalability by setting up the player to load content resources directly from the S3 bucket. For this you must grant read access on the bucket to anonymous users. If you have content that must not be accessible to the public (for e.g. copyright reasons), this is probably not an option. This currently only works for the player, not for the editor. Because of this you must still serve the 'get content file' route to make sure the editor can work with resources correctly. Steps: 1. Grant read-only permission to anonymous users for your bucket with bucket policies. See the [AWS documentation for details](https://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html#example-bucket-policies-use-case-2). 2. Set the configuration option `contentFilesUrlPlayerOverride` to point to your S3 bucket. The URL must also include the contentID of the object. For this, you must add the placeholder `{{contentId}}` to the configuration value. Examples: ```typescript contentFilesUrlPlayerOverride = 'https://bucket.s3server.com/{{contentId}}'; // or contentFilesUrlPlayerOverride = 'https://s3server.com/bucket/{{contentId}}'; ``` ## Developing and testing There are automated tests in [`/test/implementation/db/MongoS3ContentStorage.test.ts`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-mongos3/test/MongoS3ContentStorage.test.ts). However, these tests will not be called automatically when you run `npm run test` or other test calls. The reason is that the tests require a running MongoDB and S3 instance and thus need more extensive setup. To manually execute the tests call `npm run test:h5p-mongos3`. To quickly get a functioning MongoDB and S3 instance, you can use the [Docker Compose file in the scripts directory](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/scripts/mongo-s3-docker-compose.yml) like this (you obviously must install [Docker](https://docs.docker.com/engine/install/) and [Docker Compose](https://docs.docker.com/compose/install/) first): ```bash docker-compose -f scripts/mongo-s3-docker-compose.yml up -d ``` This will start a MongoDB server and MinIO instance in containers. Note that the instances will now be started when your system boots. To stop them from doing this and completely wipe all files from your system, execute: ```bash docker-compose -f scripts/mongo-s3-docker-compose.yml down -v ``` The MinIO instance will not include a bucket by default. You can create one with the [GUI tool "S3 Browser"](https://s3browser.com/), for example, or with the AWS CLI. # S3 Temporary File Storage There is an implementation of the `ITemporaryFileStorage` interface that uses a S3-compatible storage system to store files (images, video, audio etc.) that are uploaded in the editor. These files are later copied to the permanent content storage once the users save their changes. You can find it at [/packages/h5p-mongos3/src/S3TemporaryFileStorage.ts](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-mongos3/src/S3TemporaryFileStorage.ts). **Note:** You must create the S3 bucket manually before it can be used by `S3TemporaryFileStorage`! **It's also your responsibility to configure the bucket to automatically delete old temporary files after a sensible timespan (e.g. 1 day).** ## Dependencies The implementation depends on this npm package: * aws-sdk **You must add it manually to your application using `npm install aws-sdk`!** ## Usage You must import the storage implementation via a submodule: ```typescript import { S3TemporaryFileStorage, initS3 } from '@lumieducation/h5p-mongos3'; ``` or in classic JS style: ```javascript const { S3TemporaryFileStorage, initS3 } = require('@lumieducation/h5p-mongos3'); ``` Initialize the storage implementation like this: ```typescript const temporaryStorage = new S3TemporaryFileStorage( initS3({ credentials: { accessKeyId: 's3accesskey', // optional if env. variable is set secretAccessKey: 's3accesssecret' // optional if env. variable is set }, endpoint: 'http://127.0.0.1:9000', // optional if env. variable is set region: 'us-east-1' // optional if env. variable is set forcePathStyle: true }), { s3Bucket: 'h5ptemporarybucket' } ); ``` ### Notes * The function [`initS3`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-mongos3/src/initS3.ts) creates an S3 client using the `aws-sdk` npm package. * You can pass credentials and other configuration values to `initS3` and through the function parameters. Alternatively you can use these environment variables instead of using the function parameters: * AWS\_ACCESS\_KEY\_ID * AWS\_SECRET\_ACCESS\_KEY * AWS\_S3\_ENDPOINT * AWS\_REGION * You can change the bucket `h5ptemporarybucket` to any name you want, but you must specify one. You must create the bucket manually before you can use it. * The configuration object passed into `initS3` is passed on to `aws-sdk`, so you can set any custom configuration values you want. * To achieve greater configurability, you can decide not to use `initS3` or and instantiate the required client yourself. * While Amazon S3 supports keys with up to 1024 characters, some other S3 systems such as Minio might only support less in certain situations. To cater for these system you can set the option `maxKeyLength` to the value you need. It defaults to 1024. ## Using S3TemporaryFileStorage in the example The [example Express application](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-examples/src/express.ts) can be configured to use the S3 temporary storage by setting the environment variables from above and these additional variables: * TEMPORARYSTORAGE=s3 * TEMPORARY\_AWS\_S3\_BUCKET An example call would be: ```bash TEMPORARYSTORAGE=s3 AWS_ACCESS_KEY_ID=minioaccesskey AWS_SECRET_ACCESS_KEY=miniosecret AWS_S3_ENDPOINT="http://127.0.0.1:9000" TEMPORARY_AWS_S3_BUCKET=h5ptemporarybucket npm start ``` ## Customizing permissions By default the storage implementation allows all users read access to all files and every use can create new temporary files! It is very possible that this is not something you want! You can add a function to the options object of the constructor of `S3TemporayStorage` to customize access restrictions: ```typescript getPermissions: ( userId: string, filename?: string ) => Promise; ``` The function receives the userId of the user who is trying to access a filename. It must return a list of permissions the user has on the file. Your implementation of this function will probably be an adapter that hooks into your rights and permission system. # Mongo Library Storage There is an implementation of the `ILibraryStorage` interface that stores the metadata **and files** of libraries in MongoDB. As the library files should never be above the limit of MongoDB's binary data fields (\~5 MB), we we don't use GridFS. You can find the storage class in [/packages/h5p-mongos3/src/MongoLibraryStorage.ts](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-mongos3/src/MongoLibraryStorage.ts). There is another very similar storage class [MongoS3LibraryStorage](/npm-packages/h5p-mongos3/mongo-s3-library-storage), which uses S3 to store library files and might be a better fit if you have to storage very large library files (none of the normal content types do!) or if you need to reduce load on your MongoDB server. ## Dependencies The implementation depends on this NPM package: * mongodb **You must add it manually to your application using `npm install mongodb`!** ## Usage You must import the storage implementation: ```typescript import { MongoLibraryStorage, initMongo } from '@lumieducation/h5p-mongos3'; ``` or in classic JS style: ```javascript const { MongoLibraryStorage, initMongo } = require('@lumieduation/h5p-mongos3'); ``` Initialize the storage implementation like this: ```typescript const storage = new MongoLibraryStorage( ( await initMongo( 'mongodb://127.0.0.1:27017', // optional if env. variable is set 'testdb1', // optional if env. variable is set 'root', // optional if env. variable is set 'h5pnodejs' // optional if env. variable is set ) ).collection('h5p') ); await storage.createIndexes(); ``` You can safely call `createIndexes()` every time you start you application, as MongoDB checks if indexes already exist before it creates new ones. ### Notes * The function [`initMongo`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-mongos3/src/initMongo.ts) creates a MongoDB client using the `mongodb` npm package. * You can pass credentials and other configuration values to `initMongo` through the function parameters. Alternatively you can use these environment variables instead of using the function parameters: * MONGODB\_URL * MONGODB\_DB * MONGODB\_USER * MONGODB\_PASSWORD * You can change the MongoDB collection name `h5p` to any name you want. If the collection doesn't exist yet, it will be automatically created. * To achieve greater configurability, you can decide not to use `initMongo` and instantiate the required clients yourself. ## Using MongoLibraryStorage in the example The [example Express application](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-examples/src/express.ts) can be configured to use the MongoDB library storage by setting the environment variables from above and these additional variables: * LIBRARYSTORAGE=mongo * LIBRARY\_MONGO\_COLLECTION An example call would be: ```bash MONGODB_URL="mongodb://127.0.0.1:27017" MONGODB_DB=testdb1 MONGODB_USER=root MONGODB_PASSWORD=h5pnodejs LIBRARYSTORAGE=mongo LIBRARY_MONGO_COLLECTION=h5p npm start ``` ## Migrations The method `MongoLibraryStorage.migrate` can be called when you move to a new major version of MongoLibraryStorage: ```ts await storage.migrate(/*from major version*/ 9, /*to major version*/ 10); ``` Calling this method will migrate the MongoDB collection data. Note that there is no versioning of the MongoDB collection inside MongoDB. It's your job to decide when to call the migration! ### Currently supported migrations * v9 to v10: Introduces new field ubername that has the same value as \_id ## Developing and testing There are automated tests in [`/test/implementation/db/MongoLibraryStorage.test.ts`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-mongos3/test/MongoLibraryStorage.test.ts). However, these tests will not be called automatically when you run `npm run test` or other test calls. The reason is that the tests require a running MongoDB and S3 instance and thus need more extensive setup. To manually execute the tests call `npm run test:h5p-mongos3`. To quickly get a functioning MongoDB instance, you can use the [Docker Compose file in the scripts directory](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/scripts/mongo-s3-docker-compose.yml) like this (you obviously must install [Docker](https://docs.docker.com/engine/install/) and [Docker Compose](https://docs.docker.com/compose/install/) first): ```bash docker-compose -f scripts/mongo-s3-docker-compose.yml up -d ``` This will start a MongoDB server and MinIO instance in containers. Note that the instances will now be started when your system boots. To stop them from doing this and completely wipe all files from your system, execute: ```bash docker-compose -f scripts/mongo-s3-docker-compose.yml down -v ``` # Mongo/S3 Library Storage There is an implementation of the `ILibraryStorage` interface that stores the metadata of libraries in MongoDB and the files in S3. You can find the storage class in [/packages/h5p-mongos3/src/MongoS3LibraryStorage.ts](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-mongos3/src/MongoS3LibraryStorage.ts). There is another very similar storage class [MongoLibraryStorage](/npm-packages/h5p-mongos3/mongo-library-storage), which doesn't use S3 and might be better in many use cases, in particular if you S3 service is too slow or if you pay per request. ## Dependencies The implementation depends on these npm packages: * aws-sdk * mongodb **You must add them manually to your application using `npm install aws-sdk mongodb`!** ## Usage You must import the storage implementation: ```typescript import { MongoS3LibraryStorage, initS3, initMongo } from '@lumieducation/h5p-mongos3'; ``` or in classic JS style: ```javascript const { MongoS3LibraryStorage, initS3, initMongo } = require('@lumieduation/h5p-mongos3'); ``` Initialize the storage implementation like this: ```typescript const storage = new MongoLibraryStorage( initS3({ credentials: { accessKeyId: 's3accesskey', // optional if env. variable is set secretAccessKey: 's3accesssecret' // optional if env. variable is set }, endpoint: 'http://127.0.0.1:9000', // optional if env. variable is set region: 'us-east-1', // optional if env. variable is set forcePathStyle: true }), ( await initMongo( 'mongodb://127.0.0.1:27017', // optional if env. variable is set 'testdb1', // optional if env. variable is set 'root', // optional if env. variable is set 'h5pnodejs' // optional if env. variable is set ) ).collection('h5p'), { s3Bucket: 'h5plibrarybucket' } ); await storage.createIndexes(); ``` You can safely call `createIndexes()` every time you start you application, as MongoDB checks if indexes already exist before it creates new ones. ### Notes * The function [`initS3`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-mongos3/src/initS3.ts) creates an S3 client using the `aws-sdk` npm package. * The function [`initMongo`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-mongos3/src/initMongo.ts) creates a MongoDB client using the `mongodb` npm package. * You can pass credentials and other configuration values to `initMongo` through the function parameters. Alternatively you can use these environment variables instead of using the function parameters: * AWS\_ACCESS\_KEY\_ID * AWS\_SECRET\_ACCESS\_KEY * AWS\_S3\_ENDPOINT * AWS\_REGION * MONGODB\_URL * MONGODB\_DB * MONGODB\_USER * MONGODB\_PASSWORD * You can change the MongoDB collection name `h5p` to any name you want. If the collection doesn't exist yet, it will be automatically created. * You can change the bucket `h5plibrarybucket` to any name you want, but you must specify one. You must create the bucket manually before you can use it. * The configuration object passed into `initS3` is passed on to `aws-sdk`, so you can set any custom configuration values you want. * To achieve greater configurability, you can decide not to use `initS3` or `initMongo` and instantiate the required clients yourself. * While Amazon S3 supports keys with up to 1024 characters, some other S3 systems such as Minio might only support less in certain situations. To cater for these system you can set the option `maxKeyLength` to the value you need. It defaults to 1024. ## Using MongoLibraryStorage in the example The [example Express application](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-examples/src/express.ts) can be configured to use the MongoDB library storage by setting the environment variables from above and these additional variables: * LIBRARYSTORAGE=mongos3 * LIBRARY\_MONGO\_COLLECTION * LIBRARY\_MONGO\_COLLECTION * LIBRARY\_AWS\_S3\_BUCKET An example call would be: ```bash MONGODB_URL="mongodb://127.0.0.1:27017" MONGODB_DB=testdb1 MONGODB_USER=root MONGODB_PASSWORD=h5pnodejs LIBRARYSTORAGE=mongos3 LIBRARY_MONGO_COLLECTION=h5p LIBRARY_AWS_S3_BUCKET=h5plibrarybucket npm start ``` ## Migrations The method `MongoLibraryStorage.migrate` can be called when you move to a new major version of MongoLibraryStorage: ```ts await storage.migrate(/*from major version*/ 9, /*to major version*/ 10); ``` Calling this method will migrate the MongoDB collection data. Note that there is no versioning of the MongoDB collection inside MongoDB. It's your job to decide when to call the migration! ### Currently supported migrations * v9 to v10: Introduces new field ubername that has the same value as \_id ## Developing and testing There are automated tests in [`/test/implementation/db/MongoS3LibraryStorage.test.ts`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-mongos3/test/MongoS3LibraryStorage.test.ts). However, these tests will not be called automatically when you run `npm run test` or other test calls. The reason is that the tests require a running MongoDB and S3 instance and thus need more extensive setup. To manually execute the tests call `npm run test:h5p-mongos3`. To quickly get a functioning MongoDB instance, you can use the [Docker Compose file in the scripts directory](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/scripts/mongo-s3-docker-compose.yml) like this (you obviously must install [Docker](https://docs.docker.com/engine/install/) and [Docker Compose](https://docs.docker.com/compose/install/) first): ```bash docker-compose -f scripts/mongo-s3-docker-compose.yml up -d ``` This will start a MongoDB server and MinIO instance in containers. Note that the instances will now be started when your system boots. To stop them from doing this and completely wipe all files from your system, execute: ```bash docker-compose -f scripts/mongo-s3-docker-compose.yml down -v ``` # h5p-webcomponents This package provides plain HTML 5 Web Components that you can use in your project to insert H5P players or editors without having to worry about the details of setting up H5P, loading scripts and styles etc. **They only work in conjunction with a custom H5P server (e.g. one using** [**@lumieducation/h5p-server**](https://www.npmjs.com/package/@lumieducation/h5p-server)**)** that provides endpoints that do these things: * get the required data about content (one for playing, one for editing) * save content created in the editor * serve all AJAX endpoints required by the H5P core It is recommended to checkout the rest example that uses @lumieducation/h5p-server to see how the component can be used in an application. If you are looking for a solution to get the H5P player working without any server-side components, you should check out [h5p-standalone by Tunapanda](https://github.com/tunapanda/h5p-standalone). ## Browser support The components deliberately avoid using component libraries like Polymer in order to reduce the number of third party dependencies of your project. However, this also means that your users need to use a browser that supports all the web component features used in this library (ES6 classes, window\.customElements and custom HTML templates). As of December 2020 all major browsers that are still supported by their vendors will work, but if you still need to support legacy browsers like IE 11 or older Edge versions (before it was based on Chromium), you will have to use polyfills to get the web components to work. You also have to transpile the library into ES5. See [this page](https://www.webcomponents.org/polyfills) for more details. The component also uses some modern JavaScript features like `async/await` and optional chaining. These features are supported by all cutting-edge modern browsers as of December 2020. Use a transpiler if you have to support older browsers. ## Usage Install the component with npm or yarn: ``` $ npm install @lumieducation/h5p-webcomponents ``` Then, import the component in your JavaScript code and register the h5p-player or h5p-editor tag globally: ```js import { H5PPlayerComponent, H5PEditorComponent } from '@lumieducation/h5p-webcomponents'; window.customElements.define('h5p-player', H5PPlayerComponent); window.customElements.define('h5p-editor', H5PEditorComponent); ``` There is also a convenience function that you can use instead of importing the components manually: ```js import { defineElements } from '@lumieducation/h5p-webcomponents'; defineElements('h5p-player'); // only registers the player component defineElements('h5p-editor'); // only registers the editor component defineElements('h5p-player', 'h5p-editor'); // registers player and editor component defineElements(); // registers player and editor component ``` Then, you can insert the components into your DOM: **Option 1:** Insert it directly in HTML markup: ```html ``` and later set the `loadContentCallback` (and saveContentCallback for the editor) attribute from your JavaScript code. ```js document.getElementById('player').loadContentCallback = async (contentId) => { /** retrieve content model from server and return it as Promise **/ }; document.getElementById('editor').loadContentCallback = async (contentId) => { /** retrieve content model from server and return it as Promise **/ }; document.getElementById('editor').saveContentCallback = async (contentId, requestBody) => { /** save content on server **/ }; ``` **Option 2:** Create the component in JavaScript: ```js const h5pPlayer = document.createElement('h5p-player'); h5pPlayer.setAttribute('content-id', 'XXXX'); h5pPlayer.loadContentCallback = async (contentId) => { /** retrieve content model from server and return it as Promise **/ }; someOtherElement.appendChild(h5pPlayer); const h5pEditor = document.createElement('h5p-editor'); h5pEditor.setAttribute('content-id', 'XXXX'); h5pEditor.loadContentCallback = async (contentId) => { /** retrieve content model from server and return it as Promise **/ }; h5pEditor.saveContentCallback = async (contentId, requestBody) => { /** save content on server **/ }; someOtherElement.appendChild(h5pEditor); ``` ## Initializing and refreshing content The components automatically (re-)load data from the server by calling `loadContentCallback` that you must set as a property of the DOM element (see above for examples). The components won't work without the callback. The content is automatically loaded from the server after **both** of these conditions have been fulfilled (in any order): 1. `loadContentCallback` is set 2. `content-id` is set You can change the value of `content-id` and it will discard the old content and display the new one. You can also safely remove the component from the DOM. If you want to use the editor to create new content, you can set the `content-id` to the string `new`. **Caution: If the newly created content is saved later, then its `contentId` will be `undefined` in `saveContententCallback`, not `new`!** ## Saving content in the editor To save the user's changes in the editor, you have to execute the component's `save()` method: ```js try { const { contentId, metadata } = await h5pEditor.save(); // do something with the return values or ignore them } catch error { // report errors } ``` The component will perform some client-side validation of the entered data and call `saveContentCallback` (see below) with all the data needed to save the content. ## Callbacks You must provide these callbacks for the components to work: ### H5PPlayerComponent #### loadContentCallback ```ts loadContentCallback = async (contentId: string) => Promise /** see types.ts in @lumieducation/h5p-server for details how IPlayerModel looks like **/ ``` You have to set `loadContentCallback` to a function that retrieves the necessary data from the backend. It returns a promise of data that follows the structure of IPlayerModel in [types.ts](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-server/src/types.ts) in @lumieducation/h5p-server. If there is an error, the callback should throw an error object with the error message in the `message` property. If you use @lumieducation/h5p-server you will get the necessary information by using a renderer that simply returns the player model if you call `H5PPlayer.render(...)`: ```ts h5pPlayerOnServer.setRenderer(model => model); const playerModel = await h5pPlayerOnServer.render(contentId, user); // send playerModel to client and return it in loadContentCallback ``` ### H5PEditorComponent #### loadContentCallback ```ts loadContentCallback = async (contentId?: string) => Promise< IEditorModel /** see types.ts in @lumieducation/h5p-server for details **/ & { library?: string; metadata?: IContentMetadata; params?: any; }> ``` This callback is executed when the component needs to load data for a content id. The callback must create a request to an endpoint on the server, which retrieves all necessary information. The server-side implementation of the endpoint using @lumieducation/h5p-server has to combine the results of H5PEditor.render(...) and H5PEditor.getContent(...). The render must be set to simply return the editor model like this: ```js h5pEditorOnServer.setRenderer(model => model); ``` Notes: * `contentId` can be `undefined` if the editor is used to create new content * The library, metadata and params property of the returned object must only be defined if `contentId` is defined. * The callback should throw an error with a message in the message property if something goes wrong. #### saveContentCallback ```ts saveContentCallback = async ( contentId: string, requestBody: { library: string; params: { params: any; metadata: any; } ) => Promise<{ contentId: string; metadata: IContentMetadata }> ``` This callback is executed when the editor was told to save its content. You have to reach out to the server and persist the changes. When using @lumieducation/h5p-server, the server-side endpoint should call `H5PEditor.saveOrUpdateContentReturnMetaData(...)` and then return the result to the client, which returns the result as the return value of `saveContentCallback`. Note: `contentId` can be `undefined`, if the user is creating new content. ## Events The components emit a few events using the standard `dispatchEvent` method of the component. You can listen to them by adding an event listener with `addEventListener`. ### H5PPlayerComponent #### initialized This event is emitted by this component when the H5P player has initialized. The event includes the `contentId` in `event.detail.contentId`. Note: The event is emitted by H5P at a point when all the scripts but possibly not all resources (images etc.) might be loaded. You should only use the event as an indication of initialization state. #### xAPI This is an event that is emitted by this component when an action is triggered on or inside the H5P content through the H5P player. The event includes the `statement` in `event.detail.statement`,`context` in `event.detail.context`, and `event` in `event.detail.event`. ### H5PEditorComponent #### editorloaded This event is emitted when the H5P editor has been fully loaded and the editor is usable. It is advisable to block user interaction with the component until this event has been fired. The event includes the `contentId` in `event.detail.contentId` and an ubername of the main content type in `event.detail.ubername`. #### saved This event is emitted when saving was successful. The respective `contentId` can be found in `event.detail.contentId` and the content metadata in `event.detail.metadata`. Note: You can also simply use the return value of the `save()` method instead of subscribing to this event. #### save-error This event is emitted when there was an error while saving the content. A more detailed message can be found in `event.detail.message`. Note: You can also simply catch errors by wrapping the `save()` method in a `try {...} catch {...}` block instead of subscribing to this event. ## Executing underlying H5P functionality The H5PPlayerComponent offers properties and methods that can be used to do things with the underlying "core" H5P data structures and objects: ### h5pInstance This property is the object found in H5P.instances for the contentId of the object. Contains things like the parameters of the content, its metadata and structures created by the content type's JavaScript. **The object is only available after the `initialized` event was fired. Important: This object is only partially typed and there are more properties and methods on it!** ### h5pObject H5P has a global "H5P" namespace that is often used like this: ```ts H5P.init(); // initialize H5P H5P.externalDispatcher.on('xAPI', myCallback); const dialog = new H5P.Dialog(...); ``` The problem you'll face when you try to use this namespace is that you typically want to operate on the object inside the H5P iframe if a content type requires iframe embedding. The h5pObject property solves this problem: It contains the correct global H5P namespace regardless of whether there's an iframe or not. You can use it like this: ```ts const H5Pns = myPlayerComponent.h5pObject; H5Pns.externalDispatcher.on('xAPI', myCallback); const dialog = new H5Pns.Dialog(...); ``` **The property is only available after the `initialized` event was fired. Important: This object is only partially typed and there are more properties and methods on it!** ### getCopyrightHtml You can get the copyright notice of the content by calling this method. Returns undefined if there is not copyright information. Returns HTML code that you must display somewhere. ### hasCopyrightInformation Returns true if there is copyright information to be displayed. ### resize Signals the H5P content inside the component to resize itself according to the dimensions of the container. Can be called if you have changed the size of the container in an unusual fashion (e.g. by scaling it with CSS transform) and you need to resize the content manually. The component already automatically calls the resize function when its own size is changed, so this will rarely be the case. ### showCopyright Shows the copyright information in a window overlaying the H5P content. ## Support This work obtained financial support for development from the German BMBF-sponsored research project "lea.online -" (FKN: 41200147). Read more about them at the following websites: * lea.online Blog (German) - blogs.uni-bremen.de/leaonline * University of Bremen - * BMBF - # h5p-react This package provides two React components that you can use to display H5P players and editors in React applications without having to worry about the details of setting up H5P, loading scripts and styles etc. They are simple wrappers around the h5p-webcomponents package that allow you to use the Web Components in a native React style (passing in props) without having to register native event handlers etc. **The components only work in conjunction with a custom H5P server (e.g. one using** [**@lumieducation/h5p-server**](https://www.npmjs.com/package/@lumieducation/h5p-server)**)** that provides endpoints that do these things: * get the required data about content (one for playing, one for editing) * save content created in the editor * serve all AJAX endpoints required by the H5P core It is recommended to checkout the rest example that uses @lumieducation/h5p-server to see how the component can be used in an application. If you are looking for a solution to get the H5P player working without any server-side components, you should check out [h5p-standalone by Tunapanda](https://github.com/tunapanda/h5p-standalone). ## Browser support The components deliberately avoid using component libraries like Polymer in order to reduce the number of third party dependencies of your project. However, this also means that your users need to use a browser that supports all the web component features used in this library (ES6 classes, window\.customElements and custom HTML templates). As of December 2020 all major browsers that are still supported by their vendors will work, but if you still need to support legacy browsers like IE 11 or older Edge versions (before it was based on Chromium), you will have to use polyfills to get the web components to work. You also have to transpile the library into ES5. See [this page](https://www.webcomponents.org/polyfills) for more details. The component also uses some modern JavaScript features like `async/await` and optional chaining. These features are supported by all cutting-edge modern browsers as of December 2020. Use a transpiler if you have to support older browsers. ## Usage Install the component with npm or yarn: ``` $ npm install @lumieducation/h5p-react ``` Then, import the component in your JavaScript / TypeScript code: ```js import { H5PPlayerUI, H5PEditorUI } from '@lumieducation/h5p-react'; ``` Then, you can insert the components into your JSX: ```jsx { /** retrieve content model from server and return it as Promise **/ } } /> { /** retrieve content model from server and return it as Promise **/} } saveContentCallback = { async (contentId, requestBody) => { /** save content on server **/ } }/> ``` ## Initializing and refreshing content The components automatically (re-)load data from the server by calling `loadContentCallback`. The components won't work without the callback. The content is automatically loaded from the server after **both** of these conditions have been fulfilled (in any order): 1. `loadContentCallback` is set or changed 2. `contentId` is set You can change the value of `contentId` and it will discard the old content and display the new one. You can also safely remove the component from the DOM. If you want to use the editor to create new content, you can set the `contentId` to the string `new`. **Caution: If the newly created content is saved later, then its `contentId` will be `undefined` in `saveContententCallback`, not `new`!** ## Saving content in the editor To save the user's changes in the editor, you have to execute the component's `save()` method (you must get a reference to component by using `React.createRef`): ```js const h5pEditor = React.createRef(); // set ref={h5pEditor} on try { const { contentId, metadata } = await h5pEditor.save(); // do something with the return values or ignore them } catch error { // report errors } ``` The component will perform some client-side validation of the entered data and call `saveContentCallback` (see below) with all the data needed to save the content. ## Callbacks You must provide these callbacks in the props for the components to work: ### H5PPlayerUI #### loadContentCallback ```ts loadContentCallback = async (contentId: string) => Promise /** see types.ts in @lumieducation/h5p-server for details how IPlayerModel looks like **/ ``` You have to set `loadContentCallback` to a function that retrieves the necessary data from the backend. It returns a promise of data that follows the structure of IPlayerModel in [types.ts](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-server/src/types.ts) in @lumieducation/h5p-server. If there is an error, the callback should throw an error object with the error message in the `message` property. If you use @lumieducation/h5p-server you will get the necessary information by using a renderer that simply returns the player model if you call `H5PPlayer.render(...)`: ```ts h5pPlayerOnServer.setRenderer((model) => model); const playerModel = await h5pPlayerOnServer.render(contentId, user); // send playerModel to client and return it in loadContentCallback ``` ### H5PEditorUI #### loadContentCallback ```ts loadContentCallback = async (contentId?: string) => Promise< IEditorModel /** see types.ts in @lumieducation/h5p-server for details **/ & { library?: string; metadata?: IContentMetadata; params?: any; }> ``` This callback is executed when the component needs to load data for a content id. The callback must create a request to an endpoint on the server, which retrieves all necessary information. The server-side implementation of the endpoint using @lumieducation/h5p-server has to combine the results of H5PEditor.render(...) and H5PEditor.getContent(...). The render must be set to simply return the editor model like this: ```js h5pEditorOnServer.setRenderer((model) => model); ``` Notes: * `contentId` can be `undefined` if the editor is used to create new content * The library, metadata and params property of the returned object must only be defined if `contentId` is defined. * The callback should throw an error with a message in the message property if something goes wrong. #### saveContentCallback ```ts saveContentCallback = async ( contentId: string, requestBody: { library: string; params: { params: any; metadata: any; } ) => Promise<{ contentId: string; metadata: IContentMetadata }> ``` This callback is executed when the editor was told to save its content. You have to reach out to the server and persist the changes. When using @lumieducation/h5p-server, the server-side endpoint should call `H5PEditor.saveOrUpdateContentReturnMetaData(...)` and then return the result to the client, which returns the result as the return value of `saveContentCallback`. Note: `contentId` can be `undefined`, if the user is creating new content. ## Event Handlers The components emit a few events, to which you cna subscribe using the `on...` methods in the props of the components. ### H5PPlayerUI #### onInitialized This event handler is called when the H5P player has initialized. The event includes the `contentId` in the parameters. Note: The event is emitted by H5P at a point when all the scripts, but possibly not all resources (images etc.) might be loaded. You should only use the event as an indication of initialization state. ### onxAPIStatement This event handler is called when the H5P content fires an xAPI statement. Use it to collect information about what the user does with your content. ### H5PEditorUI #### onLoaded This event handler is called when the H5P editor has been fully loaded and the editor is usable. It is advisable to block user interaction with the component until this event has been fired. The event includes the `contentId` and an ubername of the main content type in the paramters. #### onSaved This event handler is called when saving was successful. The respective `contentId` and content metadata can be found in the paramters. Note: You can also simply use the return value of the `save()` method instead of subscribing to this event. #### onSaveError This event handler is called when there was an error while saving the content. A more detailed message can be found in the `message` paramter. Note: You can also simply catch errors by wrapping the `save()` method in a `try {...} catch {...}` block instead of subscribing to this event. ## Executing underlying H5P functionality The H5PPlayerComponent offers properties and methods that can be used to do things with the underlying "core" H5P data structures and objects: ### h5pInstance This property is the object found in H5P.instances for the contentId of the object. Contains things like the parameters of the content, its metadata and structures created by the content type's JavaScript. **The object is only available after the `initialized` event was fired. Important: This object is only partially typed and there are more properties and methods on it!** ### h5pObject H5P has a global "H5P" namespace that is often used like this: ```ts H5P.init(); // initialize H5P H5P.externalDispatcher.on('xAPI', myCallback); const dialog = new H5P.Dialog(...); ``` The problem you'll face when you try to use this namespace is that you typically want to operate on the object inside the H5P iframe if a content type requires iframe embedding. The h5pObject property solves this problem: It contains the correct global H5P namespace regardless of whether there's an iframe or not. You can use it like this: ```ts const H5Pns = myPlayerComponent.h5pObject; H5Pns.externalDispatcher.on('xAPI', myCallback); const dialog = new H5Pns.Dialog(...); ``` **The property is only available after the `initialized` event was fired. Important: This object is only partially typed and there are more properties and methods on it!** ### getCopyrightHtml You can get the copyright notice of the content by calling this method. Returns undefined if there is not copyright information. Returns HTML code that you must display somewhere. ### hasCopyrightInformation Returns true if there is copyright information to be displayed. ### resize Signals the H5P content inside the component to resize itself according to the dimensions of the container. Can be called if you have changed the size of the container in an unusual fashion (e.g. by scaling it with CSS transform) and you need to resize the content manually. The component already automatically calls the resize function when its own size is changed, so this will rarely be the case. ### showCopyright Shows the copyright information in a window overlaying the H5P content. # h5p-redis-lock This package provides a lock mechanism that can be used if the library runs in multi-process or cluster-mode. The locks are needed to avoid race conditions when installing libraries. ```ts import { createClient } from '@redis/client'; import { H5PEditor, H5PPlayer } from '@lumieducation/h5p-server'; import RedisLockProvider from '@lumieducation/h5p-redis-lock'; // Create a regular redis connection const redisClient = createClient({ socket: { port, host, }, database }); try { await redisClient.connect(); } catch (error) { // handle error } // Create the lock provider const lockProvider = new RedisLockProvider(redisClient); // Pass it to the editor and player object const h5pEditor = new H5PEditor( /*other parameters*/, options: { lockProvider } ); const h5pPlayer = new H5PPlayer( /*other parameters*/, options: { lockProvider } ); ``` It is important to make sure that all instances of H5PEditor use a redis lock provider that points to the same database. Otherwise race conditions can happen. # h5p-svg-sanitizer ## Background SVGs can contain malicious JavaScript code as, for instance, explained in [this article](https://vnbrs.medium.com/a-lesser-known-vector-for-xss-attacks-svg-files-d700345fff1d). That's why this library doesn't allow uploading SVG files as part of the content of H5P packages or media files (IH5PConfig.contentWhitelist doesn't contain the svg extension by default). SVG uploads as parts of libraries files, e.g. icons are still allowed, as these can only be uploaded by privileged users who can upload executable JavaScript files anyway, so uploading SVG files with injected scripts doesn't open a new attack vector. If you want to allow SVG files in content and still be protected against XSS attacks, you can use this SVG sanitizer package. The sanitizer relies on the [dompurify package](https://www.npmjs.com/package/dompurify) to do the actual sanitization. ## Usage 1. Install the `@lumieducation/h5p-svg-sanitizer` package in your application. 2. Add the sanitizer to the H5P editor object options and add the `svg` extension to the `contentWhitelist` property of the H5P configuration: ```ts const h5pEditor = new H5PEditor( cache, { ... // We've added svg to the whitelist! contentWhitelist: 'svg json png jpg jpeg gif bmp tif tiff eot ttf woff woff2 otf webm mp4 ogg mp3 m4a wav txt pdf rtf doc docx xls xlsx ppt pptx odt ods odp xml csv diff patch swf md textile vtt webvtt gltf glb', ... }, libraryStorage, contentStorage, temporaryStorage, translationCallback, urlGenerator, { // Add the sanitizer fileSanitizers: [new SvgSanitizer()] }); ``` Now there is protection against XSS in SVGs in these cases: * Uploading individual SVG media files (via `H5PEditor.saveContentFile`) * Uploading H5P packages with SVG files in the `content` directory from the GUI (stores media files in temporary storage) (via `ContentStorage.copyFromDirectoryToTemporary` or `H5PEditor.uploadPackage`) * Uploading H5P Packages with SVG files in the `content` directory and directly storing them in the content (via `ContentStorage.saveContentFile`) ## Example The examples in `packages/h5p-examples` and `packages/h5p-rest-example-server` already use the SVG sanitizer. **Note:** File sanitization only works of you pass uploaded content files to `H5PEditor.saveContentFile` as temporary files, not as in-memory streams. That's why we temporary file uploads are enable by default in the example. This can be disabled by setting the environment variable `TEMP_UPLOADS` to `false`. The environment variable `TEMP_UPLOADS` is part of the example code and won't work in your custom implementation, if you don't add explicit support for it. ## Caveats The SVG sanitizer mutates the uploaded SVG content files, so they are not identical to the ones originally uploaded. For example the order of element attributes might be different from the one in the original file. There is also no guarantee that it only removes actually malicious code and not other parts of the SVG that are not really harmful. In most use cases this should not be a problem, but if licensing of a certain SVG file forbids changing its code or if there are parts of files that are removed and users need them, the sanitizer package might not be right for you. ## Testing whether SVGs are correctly sanitized Get the [`SVG XSS injection demo file`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-svg-sanitizer/test/xss-svg.h5p) from the repo, upload it to your system and save. You should see a simple H5P Blanks activity with the image of a gray circle. Copy the URL of the image, paste it into your browser's address bar and load it. You should now *NOT* see a popup message which is caused by executing JavaScript code in the SVG. If you see the message, something is misconfigured. # h5p-clamav-scanner This package implements the `IFileMalwareScanner` malware scanning interface of the @lumieducation/h5p-server package by calling a [ClamAV](https://www.clamav.net/) scanner. ClamAV can be either installed on the host, called through a UNIX socket or a TCP socket. The package is a light wrapper around the NPM package [clamscan](https://www.npmjs.com/package/clamscan). ## ClamAV's security level ClamAV's detection rate [is really bad](https://anti-malware-alliance.org/2024/10/04/clamav-how-effective-it-is-a-look-into-its-detection-rate/). You'd be well advised to find some other antivirus software that you can use that has a higher detection rate. Sadly, this nearly always means you have to use a paid cloud-based scanning service, which might not be an option for you. So if you aren't able to use another antivirus system, using ClamAV probably is still better than nothing. If you have another anti-virus scanner, you can use the h5p-clamav-scanner package as a base for implementing the `IFileMalwareScanner` interface. We're interested in pull requests with other implementations! ## Usage ```ts import ClamAVScanner from '@lumieducation/clamav-scanner'; // There is no public constructor as the initialization is async. // That's why we have to use an async factory method. const clamAVScanner = await clamAVScanner.create(); const h5pEditor = new H5PEditor( // ... regular configuration ... // Add the scanner to the options parameter { malwareScanners: [ clamAVScanner ] } ); ``` ## Configuration You can configure the module in code or through environment variables. Environment variables take precedence over configuration in code. See the [clamscan docs](https://www.npmjs.com/package/clamscan) for more information about the configuration. ### In Code You can specify options in the factory: ```ts const clamAVScanner = await clamAVScanner.create({ clamdscan: { host: 'clamav-hostname', port: 3310 } }); ``` ### Through environment variables You can also set options by setting these environment variables: General options: * **CLAMSCAN\_SCAN\_LOG**: Path to a writeable log file to write scan results into * **CLAMSCAN\_DEBUG\_MODE**: Whether or not to log ClamAV's info/debug/error msgs to the console * **CLAMSCAN\_PREFERENCE**: clamscan or clamdscan (if clamdscan is configured, it will always be preferred) To use a local `clamscan` binary: * **CLAMSCAN\_PATH**: Path to clamscan binary on your server * **CLAMSCAN\_DB**: Path to a custom virus definition database * **CLAMSCAN\_SCAN\_ARCHIVES**: If true, scan archives (ex. zip, rar, tar, dmg, iso, etc...) * **CLAMSCAN\_ACTIVE**: If true, this module will consider using the clamscan binary To use `clamdscan` with UNIX socket or TCP: * **CLAMDSCAN\_SOCKET**: Socket file for connecting via TCP * **CLAMDSCAN\_HOST**: IP of host to connect to TCP interface * **CLAMDSCAN\_PORT**: Port of host to use when connecting via TCP interface * **CLAMDSCAN\_TIMEOUT**: Timeout for scanning files in ms * **CLAMDSCAN\_LOCAL\_FALLBACK**: Use local preferred binary to scan if socket/tcp fails * **CLAMDSCAN\_PATH**: Path to the clamdscan binary on your server * **CLAMDSCAN\_CONFIG\_FILE**: Specify config file if it's in an unusual place * **CLAMDSCAN\_MULTISCAN**: Scan using all available cores * **CLAMDSCAN\_RELOAD\_DB**: If true, will re-load the DB on every call (slow) ## Example The examples in `packages/h5p-examples` and `packages/h5p-rest-example-server` can be configured to use the ClamAV scanner class. Start the example like this: ```sh CLAMSCAN_ENABLED=true npm start ``` Note: * The `CLAMSCAN_ENABLED` environment variable is part of the example code and won't work if you don't add specific support for it. It triggers the creation of a `ClamAVScanner` instance. You can use the other environment variables to configure the `ClamAVScanner` instance as needed. * Malware scanning only works of you pass uploaded content files to `H5PEditor.saveContentFile` as temporary files, not as in-memory streams. Temporary file uploads are used by default, in the example (and could be disabled with the environment variable TEMP\_UPLOADS=false). The environment variable TEMP\_UPLOADS is part of the example code and won't work in your custom implementation, if you don't add explicit support for it. # Development # Getting started ## Prerequisites Make sure you have [`git`](https://git-scm.com/), [`node`](https://nodejs.org/) >= 20, and [`npm`](https://www.npmjs.com/get-npm) installed. There might be problems if you use `yarn` as it doesn't use the `package-lock.json` file and you might get incorrect and untested dependencies. **Important:** If you use Windows, you must use Bash (comes with Git for windows) as a command shell (otherwise scripts won't run). ## Installation for development purposes ```bash git clone https://github.com/lumieducation/h5p-nodejs-library cd h5p-nodejs-library npm install ``` This will install all dependencies in all packages, linking all cross-dependencies and downloads test dependencies such as the h5p core and editor library as well as content types. ## Structure of the repository This repository is a [NPM workspaces](https://docs.npmjs.com/cli/v7/using-npm/workspaces) monorepo. A monorepo is one repository for several packages, which can be found in the `packages/` folder. Each subfolder is its own package, published via [npm](https://www.npmjs.com). Packages are mostly self contained except for the following cases: * NPM modules needed for every package are located in the root `package.json` and `node_module` folder. For example, the [jest](https://jestjs.io) testing framework and `typescript` are used in every package - therefore these are made accessible in every package. * data used for unit and integration tests that are required by more than one package are located in `test/data`. Data used for only single packages is located in the respective `package//test/data` folder. ## Building the TypeScript files You must transpile the TypeScript files to ES5 for the project to work (the TypeScript transpiler will be installed automatically if you run `npm install`): ```bash npm run build ``` ## Running the server-side-rendering example To start the server-side-rendering example run ```bash npm start ``` and open in your browser. # Testing & code quality ## Running Tests After installation, you can run the tests with ```bash npm run test npm run test:integration npm run test:e2e npm run test:h5p-mongos3 # (require running MongoDB server) ``` You can run the e2e tests with h5p packages on your local system like this: ```bash H5P_FILES=test/data/hub-content ERROR_FILE=errors.txt npm run test:server+upload ``` ## Debugging The library emits log messages with [debug](https://www.npmjs.com/package/debug). To see those messages you have to set the environment variable `DEBUG` to `h5p:*`. There are several log levels. By default you'll only see the messages sent with the level `info`. To get the verbose log, set the environment variable `LOG_LEVEL` to debug (mind the capitalization). Example (for Linux): ```bash DEBUG=h5p:* LOG_LEVEL=debug node script.js ``` ## Other scripts Check out the many other npm scripts in [package.json](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/package.json) for other development functionality. ## Core updates Check out [this page](/development/core-updates) for more details on how to update the H5P core files when there is a new release of the H5P core or the H5P editor core. ## Code quality We aim at achieving high code quality by following these principles: * All public methods and all classes must contain JSDoc comments that contain a full documentation of the entity's function, all of its parameters and return values. * We try to follow the patterns of object oriented programming. * All code must be formatted by prettier. * All code must pass the TS Lint checks. * Every piece of functionality that is added to the library should be covered by a test. * When fixing a bug, a test proving that the bug has been fixed should be added to the project whenever possible. * All logical code branches should be covered by test except. * All tests should be part of the CI pipeline. * All code that is merged into master must pass the CI pipeline's tests. You can check whether your own code passes most of these requirements by running `npm run ci` (doesn't include tests requiring a database). # Core updates ## Introduction This library uses components of the "regular" H5P libraries to display the editor and the player. For this purpose, the JavaScript and CSS files that make up the JavasScript client (which is run in the browser) are copied over from Joubel's PHP implementation upon `npm install`. As newer content type versions typically as require a new core version, we need to regularly update the references to the core files. ## Steps 1. Change the script in `scripts/install.sh` to download the new H5P versions from GitHub 2. Change the values of these properties of the editor configuration object (e.g. [`H5PConfig.ts`](https://github.com/Lumieducation/H5P-Nodejs-library/blob/release/packages/h5p-server/src/implementation/H5PConfig.ts)): * `coreApiVersion` (put in the version H5P now uses in the downloaded core files; typically the version of h5p-editor-php-library without patch version, e.g. 1.24) * `h5pVersion` (put in the version of the PHP libraries themselves (includes patch version, e.g.: 1.24.1)) 3. Change the README to reflect the new versions (download links to GitHub) 4. Check if there are new JavaScript files in the core that are required to run the editor or the player. Add them to the `H5PPlayer.coreScripts()` or `H5PEditor.coreScripts()` methods in the respective files. 5. Run `npx ts-node packages/h5p-server/scripts/generate-supported-language-list.ts` to update the list of languages the editor supports. 6. Run all tests (including test:integration) to check if the everything still works as expected. (for example, inserting scripts might break tests) # Project Status The library is at a stage in which the major functionality of the H5P editor and player are working. You can check in the lists below, what is already implemented and what isn't. ## Finished functionality * [x] store and serve libraries / content types * [x] store and serve content * [x] create, read, update and delete operations on content * [x] manage file uploads (= images, video etc.) in temporary files * [x] decoupling of storage through interfaces * [x] provide AJAX endpoints for the editor and player * [x] backend communication with the H5P Hub * [x] register the site * [x] send usage statistics * [x] get information about content types on the hub * [x] download and install new content types / updates of content types on user request * [x] validation of packages (structural integrity and conformity of content and libraries) * [x] offers downloads for h5p packages ("exporting" content) * [x] support for copy & paste in the editor * [x] support for editor interface languages other than English * [x] addons (required to display mathematical formulas) * [x] MongoDB and S3 storage implementation for content and temporary files * [x] library administration endpoint and React UI component * [x] check permissions of users (install libraries, download h5p package, embed h5p package, create restricted, update libraries, install recommended, copy h5p?) * [x] MongoDB and S3 storage implementation for libraries * [x] Redis cache for caching * [x] catch and relay xAPI statements * [x] alter library files, semantics (allows site admins to change libraries without hacking the actual files; very useful) (published soon) * [x] filter html to prevent cross-site-scripting (XSS) vulnerabilities. * [x] add csrf tokens to AJAX POST requests * [x] storing user state in the player (for continuing later where the user left off) ## Unfinished functionality * [ ] validation of content against full library semantics (currently only text is validated) * [ ] logging & statistics generation: e.g. use of libraries by author, view of embedded content etc. (see h5p-php-library:h5p-event-base.class.php for a list of events) * [ ] provide embed route for content \[embed links can be generated but no route yet] * [ ] bundle and cache assets (aggregates all css and js files into two big files to decrease http requests; done in h5p-php-library:h5p-default-storage.class.php->cacheAssets(...)) * [ ] logging and statistics (there is a debug logger, but not one that allows you to log domain events) * [ ] mass content updates (possibly funded in the future) * [ ] option to disable H5P Hub * [ ] performance optimizations