primary key is like the primary criteria for your table: for an employee it can be the employee number. It is UNIQUE. Usually it is also not subject to changes.
Primary key always comes with an index. It is rather often a search criteria and it should be most often the join criteria.
A foreign key is like a reference. So your employee is in a department. Of course you have a different table here specifying all details about departments (name, manager etc). So in you employee table for each employee you have a reference, a department number.
To ensure the department which number you assigned, actually exists - this can be done using foreign keys. So nobody can be assigned to non-existing departments. Also: departments cannot be deleted while employee exists (this can be specified in the so called delete-rules)
btw, the department number itself is primary key for a department table.
Juliane