{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "%matplotlib inline" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n\n# 01. Preprocess EEG data\nThe `preprocessing pipeline ` pipeline runs the ICA\nalgorithm for an automatic removal of eyes and heart related artefacts.\nA report is automatically generated and can be used to correct\nand/or fine-tune the correction in each subject.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# Authors: Annalisa Pascarella \n# License: BSD (3-clause)\n\n# sphinx_gallery_thumbnail_number = 2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Import modules\nThe first step is to import the modules we need in the script. We import\nmainly from |nipype| and |ephypype| packages.\n\n.. |nipype| raw:: html\n\n nipype\n\n.. |ephypype| raw:: html\n\n ephypype\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import json\nimport pprint # noqa\nimport os\nimport os.path as op\nimport nipype.pipeline.engine as pe\n\nfrom ephypype.nodes import create_iterator, create_datagrabber\nfrom ephypype.pipelines.preproc_meeg import create_pipeline_preproc_meeg\nfrom ephypype.datasets import fetch_erpcore_dataset" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us fetch the data first. It is around 90 MB download.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import ephypype\nhome_dir = op.expanduser(\"~\")\n\nbase_path = op.join(home_dir, 'workshop')\n\ntry:\n os.mkdir(base_path)\n\nexcept OSError:\n print(\"directory {} already exists\".format(base_path))\n\ndata_path = fetch_erpcore_dataset(base_path)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Define data and variables\nLet us specify the variables that are specific for the data analysis (the\nmain directories where the data are stored, the list of subjects and\nsessions, ...) and the variables specific for the particular pipeline\n(downsampling frequency, EOG channels, cut-off frequencies, ...) in a\n:download:`json ` file.\n(if it is does work, try to go on the github page, and right-click \"Save As\" on the Raw button)\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# Read experiment params as json\nparams = json.load(open(\"params.json\"))\npprint.pprint({'general parameters': params['general']})\n\ndata_type = params[\"general\"][\"data_type\"]\nsubject_ids = params[\"general\"][\"subject_ids\"]\nNJOBS = params[\"general\"][\"NJOBS\"]\nsession_ids = params[\"general\"][\"session_ids\"]\n# data_path = params[\"general\"][\"data_path\"]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Read the parameters for preprocessing from the json file and print it\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "pprint.pprint({'preprocessing parameters': params[\"preprocessing\"]})\n\nl_freq = params[\"preprocessing\"]['l_freq']\nh_freq = params[\"preprocessing\"]['h_freq']\ndown_sfreq = params[\"preprocessing\"]['down_sfreq']\nEoG_ch_name = params[\"preprocessing\"]['EoG_ch_name']\nch_new_names = params[\"preprocessing\"]['ch_new_names']\nbipolar = params[\"preprocessing\"]['bipolar']\nmontage = params[\"preprocessing\"]['montage']\nn_components = params[\"preprocessing\"]['n_components']\nreject = params[\"preprocessing\"]['reject']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Specify Nodes\nBefore to create a |workflow| we have to create the |nodes| that define the\nworkflow itself. In this example the main Nodes are\n\n.. |workflow| raw:: html\n\n workflow\n\n.. |nodes| raw:: html\n\n nodes\n\n* ``infosource`` is a Node that just distributes values\n* ``datasource`` is a |DataGrabber| Node that allows the user to define flexible search patterns which can be parameterized by user defined inputs \n* ``preproc_eeg_pipeline`` is a Node containing the pipeline created by :func:`~ephypype.pipelines.create_pipeline_preproc_meeg`\n\n.. |DataGrabber| raw:: html\n\n DataGrabber\n\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n### Infosource\nThe ephypype function :func:`~ephypype.nodes.create_iterator` creates the\n``infosource`` node that allows to distributes values: when we need to feed\nthe different subject names into the workflow we only need a Node that can\nreceive the input and distribute those inputs to the workflow.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "infosource = create_iterator(['subject_id', 'session_id'],\n [subject_ids, session_ids])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n### DataGrabber\nThen we create a node to grab data. The ephypype function\n:func:`~ephypype.nodes.create_datagrabber`\ncreates a node to grab data using |DataGrabber| in Nipype. The DataGrabber\nInterface allows to define flexible search patterns which can be\nparameterized by user defined inputs (such as subject ID, session, etc.).\nIn this example we parameterize the pattern search with ``subject_id`` and\n``session_id``. The ``template_args`` in this node iterate upon the values in\nthe ``infosource`` node.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "template_path = 'sub-%s/ses-%s/eeg/sub-%s*ses-%s*.set'\ntemplate_args = [['subject_id', 'session_id', 'subject_id', 'session_id']]\ndatasource = create_datagrabber(data_path, template_path, template_args)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n### Preprocessing pipeline\nEphypype creates for us a pipeline which can be connected to these nodes we\ncreated. The preprocessing pipeline is implemented by the function\n:func:`~ephypype.pipelines.create_pipeline_preproc_meeg` thus to\ninstantiate this pipeline node, we import it and pass our parameters to it\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "preproc_workflow = create_pipeline_preproc_meeg(\n data_path, pipeline_name=\"preproc_eeg_pipeline\",\n l_freq=l_freq, h_freq=h_freq, n_components=n_components, reject=reject,\n EoG_ch_name=EoG_ch_name, data_type=data_type, montage=montage,\n bipolar=bipolar, ch_new_names=ch_new_names)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Specify Workflows and Connect Nodes\nNow, we create our workflow and specify the ``base_dir`` which tells nipype\nthe directory in which to store the outputs.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "preproc_pipeline_name = 'preprocessing_workflow'\n\nmain_workflow = pe.Workflow(name=preproc_pipeline_name)\nmain_workflow.base_dir = data_path" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We then connect the nodes two at a time. First, we connect the two outputs\n(``subject_id`` and ``session_id``) of the `infosourcenode` node to the\n`datagrabbernode` node. So, these two nodes taken together can grab data\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "main_workflow.connect(infosource, 'subject_id', datasource, 'subject_id')\nmain_workflow.connect(infosource, 'session_id', datasource, 'session_id')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Similarly, for the inputnode of the `pipnode`. Things will become\nclearer in a moment when we plot the graph of the workflow.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "main_workflow.connect(infosource, 'subject_id',\n preproc_workflow, 'inputnode.subject_id')\nmain_workflow.connect(datasource, 'raw_file',\n preproc_workflow, 'inputnode.raw_file')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Run workflow\nAfter we have specified all the nodes and connections of the workflow, the\nlast step is to run it by calling the ``run`` method. It\u2019s also possible to\ngenerate static graph representing nodes and connections between them by\ncalling ``write_graph`` method.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "main_workflow.write_graph(graph2use='colored') # optional" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Take a moment to pause and notice how the connections\nhere correspond to how we connected the nodes.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import matplotlib.pyplot as plt # noqa\nimg = plt.imread(op.join(data_path, preproc_pipeline_name, 'graph.png'))\nplt.figure(figsize=(6, 6))\nplt.imshow(img)\nplt.axis('off')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Finally, we are now ready to execute our workflow.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "main_workflow.config['execution'] = {'remove_unnecessary_outputs': 'false'}\n\n# Run workflow locally on 1 CPU\nmain_workflow.run(plugin='LegacyMultiProc', plugin_args={'n_procs': NJOBS})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Results\nThe output is the preprocessed data stored in the workflow directory\ndefined by ``base_dir``. Here we find the folder\n``preprocessing_workflow`` where all the results of each iteration are\nsorted by nodes. The cleaned data will be used in `compute_perp`.\n\nIt\u2019s a good rule to inspect the report file saved in the ``ica`` dir to look\nat the excluded ICA components.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import mne # noqa\nfrom ephypype.gather import get_results # noqa\n\nica_files, raw_files = get_results(main_workflow.base_dir,\n main_workflow.name, pipeline='ica')\n\nfor ica_file, raw_file in zip(ica_files, raw_files):\n print(f'*** {raw_file} ***')\n raw = mne.io.read_raw_fif(raw_file)\n ica = mne.preprocessing.read_ica(ica_file)\n ica.plot_properties(raw, picks=ica.exclude, figsize=[4.5, 4.5])\n\n # ica.plot_components()\n # ica.plot_sources(raw)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.8" } }, "nbformat": 4, "nbformat_minor": 0 }