The Cytoscape 'Layout' menu lists many layout algorithms, but the names presented there are different from the names by which these algorithms are known to layoutNetwork method. This method returns a named list in which the names are from the GUI, and the values identify the names you must use to choose an algorithms in the programmatic interface.
A named list of strings.
layoutNetwork getLayoutNames getLayoutPropertyNames getLayoutPropertyType getLayoutPropertyValue setLayoutProperties
cy <- CytoscapeConnection () layout.name.map <- getLayoutNameMapping (cy) print (head (names (layout.name.map), n=3)) #  "Inverted Self-Organizing Map Layout" "Group Attributes Layout" "MOEN Layout" print (head (as.character (layout.name.map), n=3)) #  "isom" "attributes-layout" "jgraph-moen"