They do have some linkage - not enough to be useful, but enough to be irritating.
The Visual Linker will write simple joins in SQL, and display these in the Show SQL Query. If you start editing the SQL Query manually, then SCR realizes it should stop trying to further change the query from the Visual Linker, so the Visual Linker is basically useless at this point - it doesn't reflect changes made to the SQL Query manually, and the SQL Query doesn't reflect subsequent changes made in the Visual Linker, with the exception that if a new table is added, and fields from that table are added to the report, then the SQL Query will show these in the SELECT statement. However, the new table will not be joined until you do that manually in either the FROM or WHERE clauses.
A similar logic applies to the Record Select Experts - once you start editing the SQL statement manually, SCR realizes it needs to stop playing with the WHERE clause (which is what the record selection attempts to do).
The best way around this is to put off manually editing the SQL statement until you have all the tables joined in, and all the record selection formulas done. It doesn't eliminate the problem, but it minimizes the impact it has on your productivity.