Created by: gwideman, Apr 1, 2015 8:19 pm
Revised by: gwideman, Apr 7, 2015 7:24 pm (23 revisions)

Overview

Arduino projects often involve multiple files beyond the single main .ino (or .pde) file, and included libraries. Indeed, third-party libraries and their examples can be quite complex, and to require a bit more understanding of where the Arduino build process expects files to be, and what it does with them.
Several main topics intertwine:
  • "Project structure"
    • The Arduino environment does not offer a project treeview (like other IDEs have), nor a makefile per se. Instead, the programmer views an Arduino project as having a main "something.ino" file, and possibly some other subsidiary files, with #include directives which select other files to incorporate into the finished executable.
    • Ultimately, an Arduino project is built by gcc tools like an ordinary C/C++ project. However, before that happens the Arduino tools transform arduino-style source files into a set of more-standard C/C++ files, which can then be digested by the gcc tools. Understanding that transformation and sequence is crucial for understanding complex Arduino projects.
  • Directory structure required by the Arduino tools and IDE.
    • In particular: How directory structure relates to items within the IDE menu structure
    • Where the Arduino tools look for files during build, and during download-to-board
  • For projects beyond the simplest, one feature that the Arduino IDE offers is additional "tabs", which correspond to additional files that get incorporated during the build. Details below.

Directory structures

Version

The directory layouts shown here pertain to Arduino versions >=1.5. I used version 1.6.1 and 1.6.3 specifically as a guide. The structure in the 1.0.x series was very similar, though slightly less elaborate in some areas.

User configurability

For some folders, the user may be chose different locations. Notably:
  • The Arduino install directory can be chosen during installation.
  • The user's Sketchbook folder can be changed via the File > Preferences, and that setting is stored in preferences.txt.
    • Note that Arduino stores and retrieves third-party libraries in the 'libraries' subdirectory of the Sketchbook folder. Therefore changing the setting for the path to the Sketchbook folder will hide third-party libraries, or requires that those libraries be copied or moved to the new Sketchbook folder.

Arduino installation folder

Per 1.6.1, with Teensyduino also installed
Folder
Files
IDE Menu
Description
folder01.gif [Arduino install dir]


Wndows default: C:\Program Files (x86)\Arduino\
folder02.gif drivers




someboard.inf files

Informs Windows how to install and invoke drivers for boards.
folder03.gif FTDI USB Drivers


Drivers to talk to boards that have FTDI USB controllers
folder02.gif examples

File > Examples
Example Arduino projects. Buildable, but not writeable?
folder03.gif 01.Basics



folder04.gifAnalogReadSerial
AnalogReadSerial.ino
-> AnalogSerialRead

folder04.gif...



folder03.gif...



folder03.gifTeensy


Adidtional Teensy-specific examples (in many subdirs)
folder02.gifhardware



folder03.gifarduino


Code that gets included into an Arduino project to interact with the particular board hardware and/or CPU
folder04.gifavr




boards.txt
Tools > Board
Lists target boards in this family of hardware, and provides specs that IDE uses to select appropriate core code, and program each board's hardware [1]

platform.txt

Provides macros for each step in the build process, suited to this family of hardware [1]

programmers.txt

Provides parameters for operating each variety of programmer [1]
folder05.gifbootloaders


Bootloader code; precompiled hex files
folder05.gifcores


Some core .c/.cpp/.h files common to Arduino or AVR
folder05.giffirmwares


Some .c/.cpp/.h files relating to USB and wifi
folder05.giflibraries

Sketch > Import Library
Libraries supplied and specific to this hardware
folder06.gif [somelibrary]


A particular library
folder07.gif examples

File > Examples
Examples for this library
folder05.gifvariants


.h files defining pin numbers for different Arduino models.
folder04.gifsam


Same structure as the peer avr folder
folder03.gifteensy


Teensy install adds this. Same structure as peer arduino folder

boards.txt
Tools > Board

folder04.gifavr



folder05.gifcores


Some core .c/.cpp/.h files for teensy, parallel to the arduino ones. Variants for different USB roles
folder05.giflibraries


Many libraries from PRJC, and subsidiary examples
folder03.giftools



folder04.gifavr



folder02.gifjava


Implementation (in Java) of Arduino IDE GUI app and other toolls
folder02.giflib


Java libraries for Arduino IDE and tools
folder02.giflibraries

Sketch > Import Library
Libraries (generally in C/C++) to incorporate in Arduino projects to run on Arduino
folder03.gif ...
...


folder03.gifServo
Servo.cpp


folder03.gifStepper
Stepper.cpp


folder03.gif ...
...


folder02.gif reference
*.html

Help files
folder02.gif tools


Subsidiary tools for carrying out the build process, including gcc (compiler) and related tools, and avrdude for downloading hex files to Arduino
_._._._._._._._._._._._._._._._._._._._._._._.

._._._._._._._._._._._._._.

Notes:
[1] See: https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5---3rd-party-Hardware-specification

Sketchbook folder

Repository of user's sketches (user-written programs)
Folder
Files
IDE Menu
Description
folder01.gif [User's Sketches folder]

File > Sketchbook
Folder set via File > Preferences, which is stored iin preferences.txt.
Windows default: [user home]\Documents\Arduino
folder02.gifASketch
ASketch.ino

Arduino main file for project is named with same base name as containing sketch folder.

morecode.ino

Additional "tabs", source files that build preprocessor "pastes" onto end of main .ino file
folder02.gif. . .


More sketch folders
folder02.giflibraries

Sketch > Import Library
Third party libraries
folder03.gifSomeLibrary
*.c, *.cpp, *.h etc

A library. Source files for the library
folder04.gifexamples
...
File > Examples
Example source code. Buildable, but read-only in the IDE?
folder05.gifAnExample
Example source code


folder02.gif . . .


More sketch folders
_._._._._._._._._._._._._._._._._._._._._._._.

._._._._._._._._._._._._._.

Third-Party Library (provided in zip file, with examples)

Third-party libraries may be supplied in a zip file, structured as follows:
Folder
Files
Description
folder01.gif SomeLibrary-1.23.zip


folder02.gifSomeLibrary
*.c; *.cpp; *.h
keywords.txt
license.txt

examplesfolder03.gif

Examples for this library
folder04.gifAnExample
AnExample.ino etc

folder04.gif. . .


_._._._._._._._._._._._._._._._._._._._._._._.


So far as I know, such a library can be installed either by:
  • Unzip + copy to Arduino's existing [Sketchbook folder]\libraries\ folder, or...
  • Arduino IDE library zip install procedure: Sketch > Import Library > Add Library... (which I think does the same thing).

Settings file preferences.txt

  • Stores the preferences of the File > Preferences dialog
  • Location of preferences.txt can be viewed in File > Preferences dialog.
  • On Windows, typical location:
    • C:\Users\[user]\AppData\Roaming\Arduino\preferences.txt ... (IDE version < 1.5)
    • C:\Users\[user]\AppData\Roaming\Arduino15\preferences.txt (IDE >= 1.5)

Working/Build directory (temporary)

The Arduino IDE makes use of several temporary folders:
  • New IDE source files: C:\Users\[user]\AppData\Local\Temp\untitled[random].tmp\...
  • Console output: C:\Users\[user]\AppData\Local\Temp\console[random].tmp\...
    • This is especially useful for troubleshooting the build steps and possible errors, especially if verbose output is turned on (File > Preferences > "Show verbose output").
  • Build output, including hex: C:\Users\[user]\AppData\Local\Temp\build[random].tmp\...
    • Very informative when attempting to determine how various files combine into your main sketch cpp file, and what modules are included in your finished product.
The names which the IDE composes for these folders contains a random string. The intent is perhaps to save every version for possibly later inspection, but do so in a way that doesn't tempt you to treat this build output as source for a subsequent step. You can determine which folders resulted from a recent build by sorting the Temp directory by date. Also you can search for files corresponding to the name of the sketch of interest.

Build process notes

How sketch source files corresponds to a normal C/C++ project

  • Sketch folder and main file: A sketch revolves around a sketch folder (directory), and the "main" sketch file within, whose basename matches the sketch folder name.
    • Sketch "MySketch" would be stored in folder ...\Sketches\MySketch, with a main file MySketch.ino.
    • A sketch may have additional "tabs", that is, additional xxx.ino files in the sketch folder, which get appended to the main file at build time.
    • (Ino files appear in separate tabs in the IDE. The IDE hides the '.ino' filename extension.)
  • Calling libraries: A sketch can optionally call upon one or many libraries. This requires adding '#include "something.h" ' directives in the normal C/C++ fashion, following the usual rules about angle brackets (finds system library headers) or double-quotes (looks for header file in sketch's folder, and possibly some others).
    • So... a sketch folder can contain xxx.h and xxx.c or xxx.cpp files. During build, these get compiled and linked, but are only useful if referenced by some other code via #include of their header file(s).
  • Sketch file transformation. During the build process the sketch's .ino files are translated into a .cpp file. See below.

Sketch file transformation

The build process derives a .cpp file from the sketch's .ino files, as follows:
arduino_transform.png
The figure shows the transformation, as observed in an actual Arduino installation (1.0.6 and 1.6.1). The transform is described in this Arduino doc page, though that page doesn't agree with what I observed:
  • "Tabs have one of four extensions: no extension, .c, .cpp, or .h". Not accurate. What I saw was that when the user adds a new tab in the IDE and names it "MyNewFile", the IDE creates a new file in the sketch folder, named "MyNewFile.ino". (It is true that the IDE displays the file names without their .ino extension in the IDE editor tabs.) It's these additional .ino files with names that don't match the sketch folder name which get added to the "main" .ino file which does match the folder name.
  • "Finally, the contents of the current target's main.cxx file are appended to the bottom of your sketch.". This statement seems to be false.
    • The build process does indeed find a main.cpp file, along with numerous other Arduino-supplied boilerplate files, and compile them.
    • These files all get linked together during the link process. So a sketch does indeed include an actual main.cpp file supplied by Arduino, but it does not get appended to the sketch's "main" file.
  • "When you verify a sketch, it is built in a temporary directory in the system temp directory (e.g. /tmp on the Mac). When you upload it, it is built in the applet/ subdirectory of the sketch's directory." In my tests, both Verify and Upload operations produced output in the Temp/build[random].tmp\ folder.

Sketch and main.cpp

Like with most C/C++ projects, there's a main function, here in a main.c file supplied by the Arduino environment. There are different variants for different hardware targets. The sketch file(s) and other libraries get linked to the main.c file, along with startup code which calls the main() function.
arduino_sketch_structure.png
Notes:
  • It's not clear to me why the IDE generates function prototypes for setup() and loop() into the composite MySketch.cpp file, when that file also includes Arduino.h which contains those same prototypes.

Build Directory Notes

Several features of the build directory may be useful in understanding the build process
  • The sketch source file(s) (MySketch.ino, say) gets transformed into a corresponding MySketch.cpp file.
  • Each source file, boilerplate file and included header and cpp file results in files in the build directory:
    • A something.d file, which is a text file listing this file's dependencies. (Essentially a "make" statement for this one file.)
    • A something.o file: compiled object file for input to the linker
  • If compile and link succeed, the build process yields:
    • MySketch.hex, MySketch.elf, MySketch.eep files, which are input for the uploader and programmer steps.

References

Arduino.cc docs
Arduino.cc Forum
  • Directory structure issues with Arduino IMPORTANT:
    • Calling C functions from cpp module, like main sketch, requires extern declaration
    • PATH used for compiling and linking. (And the 'utility' subfolder.)
Arduino pages at code.google.com
Third-party articles
Stackoverflow

Folder icons with indents
folder01.gif One
folder02.gif Two
folder03.gifThree
folder04.gif Four
folder05.gifFive
folder06.gifSix
folder07.gifSeven