Data Import Module

I’ve been working on a module with some advanced tools for importing data into Gibbon. Our school was previously using Blackbaud’s Education Edge/Faculty Access and just started using Gibbon this year, so it was important to migrate thousands of various records into the new system and help ensure that the data is formatted and validated before it goes into the database.

If anyone is interested in using or testing this module you can find the repository on GitHub:

Download here:

Install in the modules folder & activate in System Admin to use. Works with v12 and v13(dev).

Features: (v1.0)
• A multi-part import form for improved control over incoming data
• Performs a dry-run before importing to see the anticipated results
• Handles relational fields (eg: transforms usernames into gibbonPersonID on import)
• Filters and validates many types of data (dates, emails, urls, roles, etc)
• Assign columns to database values ad-hoc and can skip non-required columns
• Choose to Update & Insert, Update Only or Insert Only when importing
• Create a database snapshot before importing to rollback changes if needed
• Keeps import logs tracking the user who made them and the results of the import
• Remembers the column order from the last import to speed up future imports
• Choose to export an excel file with all importable columns pre-filled
• Imports defined with a flexible YML syntax

How to use:
Once Installed, go to Admin > Data Admin. This shows a list of all the types of imports currently setup (new ones will continue to be added, feel free to request something if it’s not in the list). The difference between these and regular Gibbon imports is they must have a header line in the csv file. The headers come into play once you go to step 2 where you can assign columns to fields, and optionally skip columns. Step 3 is a dry-run so you can see the results and any data formatting errors before the database is changed. If everything looks good, step 4 will complete the import into your database.

Snapshots are useful as an extra layer of database backup. Before you perform an import you can take a snapshot and if things go horribly wrong you can restore the snapshot. These aren’t meant for live systems where people might be making changes continuously, or as a standalone backup method (they store locally in the upload folder, where as a good backup method should be on a different drive).

To help with formatting spreadsheets into csv files you can open an import type and click Export Structure. This will give you a spreadsheet with pre-filled column names. You can also click Export Data to make changes to a set of data in spreadsheet form then re-import it. To do this use “Update Only” and “Same as below (skip first column)” in Step 1, and the “Use database ID field” option in Step 2. Both Export options create an .xls file which would need saved as .csv to re-import.

I recommend using it on a local or development system first. I have tested and used it quite a bit myself however there are no guarantees. This program is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. See the GNU General Public License for more details.

Feel free to post any questions or send feedback, either in this thread or email me: sandra.kuipers [at]



This looks like some amazing work. Thanks for releasing it under a GPL license, as this kind of community involvement will definitely help Gibbon grow (I know it contains some Gibbon core code, and so must be under a GPL license, but I know you could have coded it from scratch and made it proprietary).

I don’t have the cause (or time!) to test this out right now, but I know your work is always meticulous, so I would really like to host it on the Extend page of Would that be OK by you? I will list you as creator (do you want me to link you to a particular website?).

Let me know.



Sure, I’ll polish up a couple more things and let you know. Aside from the starter module stuff it is pretty much coded from scratch, but I’m happy to share it with the community : )


Wow, thanks skuipers…wish I had this a few weeks ago…keep it up!

Hiya, might be being an idiot, but none of the ‘export data structures’ work to produce an xls for me?


The web page at http://[domain]/modules/Data%20Admin/export_run.php?type=studentNotes might be temporarily down or it may have moved permanently

Ah scratch that, extensions missing :slight_smile:

BeardyDude, welcome to the Gibbon community! We definitely welcome anyone who suspects they “might be being an idiot” ; ) We’ve all be there. Glad that you got the issue sorted.

Hi all,

I’ve released v1.2.03 of Data Admin (download here, or on GitHub here), which adds a tool called Combine Similar Fields to help with duplicate user data, especially useful when importing data from other systems but also if duplicate data has accumulated over time.

Combine Similar Fields: With user-entered data it’s common to end up with a variety of details that all mean the same thing or are spelled incorrectly. This tool helps find and combine those fields. This can be particularly helpful if you’re generating reports.

Two modes: Assisted mode aims to help find matches between similar values (it can result in false positives though, which is also why there’s manual mode):

Manual mode lists all data and frequencies, letting you select and combine fields:

In each mode you can select which fields to combine and re-name them as a group.

More tools to help refine and merge duplicate data are in the works :smiley:

Also anyone working on migrating large data sets from other systems (before importing into Gibbon), be sure to check out Open Refine by Google

Sandra, Combine Similar Fields looks amazing. Will try and update our systems and have a go. Thanks!

Sandra Buenas noches, felicitacion por tu esfuerzo de facilitar las cosas a los administradores de Gibon
Tengo una pregunta por que cuando abro gibbon en español me aparecen algunos modulos como DATA ADMIN completaente en Ingles. Estuve revisando con la Traduccion de Poeditor y los terminos estan casi todos traducidos al español.
En algunos otros modulo he encontrado en el menú algunos terminos en ingles y la mayoria en espeñol-

Existe la posibilidad de convertir los términos que aún subsisten en Gibnon en Español, para que no quede una parte en ingles y otras en españoll?

Hay que pedir algun codigo de acceso a cada uno de los archivos?

Si te puedo colaborar en algo para terminar de hacer las traducciones estoy revisando en todo el sistema de POE la traduccion.


Hi Beatriz,

Much of the functionality of Data Admin is now in the core under System Admin > Import from File. These should be translatable through the Gibbon project in POEditor. Is it just Data Admin, or other modules that you’re looking to translate? Translating the additional non-core modules is something we’ve started to do in recent versions, but it’s not setup for all the modules yet, it’s something we’re adding on a case-by-case basis, as time permits.

Hi Sandra.

can the absolute path on line 48 in tools_findUsernamesProcess.php (‘/lib/PHPExcel/Classes/PHPExcel.php’) for Data Admin be updated to reflect the new location for PHPExcel in vendor folder (/vendor/phpoffice/phpexcel/Classes/PHPExcel.php).

Use of find Usernames in data admin throws an Error 500 in the webserver due to files not being located within the specified path.

Thanks for the heads up, I’ve released v1.5.01 which is v22 compatible and fixes the PHPExcel error.

Hi there,

Please I need assistance with importing user data. I tried to follow the Gibbon training on youtube but my import page seem to be clearly different from the illustration on youtube video.

This what i was expecting to see

Hi nejobest, welcome to the forums. It is interesting, you’re correct in that you should be seeing a list of the imports available. I wonder if perhaps your system does not have file permission to access this folder, so it’s unable to read the list of available imports. Can you check in System Admin > System Check to see if there are any file access issues detected? Also, be sure to check that the /uploads/imports folder has file permissions that will allow your server to read and write it.

Thanks so much for this. Please see report of my “System Check”