Level 4: Data Integrations Development

Build data pipelines that provide for local CSU data to be moved to, stored in, and/or connected to the UT Data Hub.

Skills You Might Need:

  • Familiarity with UT Data Hub structure, processes, and standards
  • Working experience of relational databases and concepts
  • Knowledge of APIs
  • Data extraction techniques
  • Data formats and transmit techniques
  • Data validation techniques
  • Quality Assurance techniques

Related Technology:

  • SQL
  • PL/PGSQL
  • Python
  • AWS technologies such as Glue, Lambda, etc.
  • ETL tools such as DataStage, Informatica, etc.
  • Alation

Possible Staff Roles:

Some CSUs might consider ensuring they have the appropriate staff to support their goals and needs with regards to data. Here are some potential roles and positions your unit might consider:

  • Data Engineer
  • DevOps Engineer
  • QA Engineer
  • Software Developer/Analyst
  • Software Engineer

General Essential Training:

  • Advanced SQL
  • PostgreSQL development
  • Data Modeling techniques
  • AWS Foundational training
  • AWS Developer training

UT Specific Essential Training:

  • Orientation to the UT Data Hub (currently in development)
  • Basic training in Alation (currently in development)