Skip to content

Commit

Permalink
Feat/comp/inputs (#2256)
Browse files Browse the repository at this point in the history
* • test_report.py:  passing all tests

* • Passes all tests!

* -

* -

* • composition.py: reorganize with #region and #enregions

* • composition.py: reorganize with #region and #enregions

* • controlmechanism.py, optimizationcontrolmechanism.py:
  - _instantiate_monitor_for_control_input_ports -> _parse_monitor_control_input_ports
  - refactored to support allow_probes option on ocm

* -

* -

* -

* • controlmechanism.py, optimizationcontrolmechanism.py:
  - _instantiate_monitor_for_control_input_ports -> _parse_monitor_control_input_ports
  - refactored to support allow_probes option on ocm

* • controlmechanism.py, optimizationcontrolmechanism.py:
  - _instantiate_monitor_for_control_input_ports -> _parse_monitor_control_input_ports
  - refactored to support allow_probes option on ocm

* -

* • composition.py:
  __init__: move controller to after add_nodes and add_linear_pathway

* -

* - test_control: only test_hanging_control_spec_outer_controller not passing

* -

* -

* -

* -

* -

* -

* • composition.py:
  _instantiate_control_projections:
     weird requirement for double-call to controller._instantiate_control_signal

* • test_paremtercomposition.py:
  restored parameter spec that causes crash ('threshold',Decision2)

* ª Attempt to fix problem with partially overlapping local and ocm control specs

- composition.py
  - _get_control_signals_for_composition:  (see 11/20/21)
      - added (but commented out change) to "if node.controller" to "if not node.controller"
      - changed append to extend
  - _instantiation_control_projection:
      - got rid of try and except double-call to controller._instantiate_control_signals
      -  outdented call to self.controller._activate_projections_for_composition at end

- controlmechanism.py:
    - _check_for_duplicates:  add warning and return duplicates

- optimizationcontrolmechanism._instantiate_control_signals:
    - add call to self.agent_rep._get_control_signals_for_composition() to get local control specs (on mechs in comp)
    - eliminate duplicates with control_signal specs on OCM
    - instantiate local + ocm control_signals

- parameterestimationcomposition.py
  - added context to various calls

* see later commit

* see later commit

* see later commit

* see later commit

* - This branch passes all tests except:
   - test_parameterestimationcomposition
   - test_composition/test_partially_overlapping_control_specs (ADDED IN THIS COMMINT)

- All relevant changes to this branch are marked as "11/21/21."
  However, most are commented out as they break other things.

- The tests above both involve local control specifications (on mechanism within a nested comp)
  and on the OCM for the outer composition, some of which are for the same nested mechs

- Both tests fail with:
   "AttributeError: 'NoneType' object has no attribute '_get_by_time_scale'" (in component.py LINE 3276)
   This may be due to a problem with context setting, since the error is because the modulation Parameter
   of the ControlProjection is returning "None" rather than "multiplicative_param" (when called with get(context)),
   whereas "multiplicative_param" is returned with a call to get() (i.e., with no context specified)

- Most of test_partially_overlapping_control_specs is passed if
   changes marked "11/21/21 NEW" in optimizationcontrolmechanism.py (LINE 1390) are implemented,
   but it does not properly route ControlProjections through parameter_CIMS (see last assert in test).
   Furthermore, test_parameterestimationcompsition fails with the mod param error, even though the
   model has similar structure (i.e., outer composition -- in this case a ParameterEstimationComposition)
   with an OCM that is given control specs that overlap with ones in a nested composition.

- There are also several other things in composition I found puzzling and tried modifying, but that cuased failures:
  - _get_control_signals_for_composition():
      - seems "if node.controller" should be "if **not** node.controller" (emphasis added just for comment)
      - "append" should be "extend"
  - _instantiate_control_projection():
      -  call to self.controller._activate_projections_for_composition (at end of method) should not be indented

* - small mods; don't impact anything relevant to prior commit message

* - small mods; don't impact anything relevant to prior commit message

* - small mods; don't impact anything relevant to prior commit message

* - finished adding formatting regions to composition.py

* -

* • composition.py:
  - rename _check_projection_initialization_status -> _check_controller_initialization_status
  - add _check_nodes_initialization_status(context=context)
    (and calls it with _check_controller_initialization_status)

* • show_graph.py:  addressed bug associated with ocm.allow_direct_probe

* • show_graph.py:  addressed bug associated with ocm.allow_direct_probe

* -

* Composition: add_controller: set METHOD as context source early

* -

* • composition.py
  retore append of control_signals in _instantiate_control_projections()

* • composition.py
  restore append of control_signals in _instantiate_control_projections()

• test_composition.py:
  add test_partially_overlapping_local_and_control_mech_control_specs_in_unnested_and_nested_comp

* • test_partially_overlapping_local_and_control_mech_control_specs_in_unnested_and_nested_comp():
  - added clear_registry() to allow names to be reused in both runs of test

* • composition.py
  docstring:  added projections entry to list of attributes
  - add_controller:  added call to _add_node_aux_components() for controller

* • composition.py
  _add_node_aux_components(): added deletion of item from aux_components if instantiated

* • composition.py
  - comment out _add_node_aux_components() (causing new failures)
  - move _instantiate_control_projections to be with _instantiate_control_projections,
       after self.add_node(self.controller.objective_mechanism (to be more orderly)

* -

* - confirm that it passes all tests exception test_composition/test_partially_overlapping...
  (with addition of _add_aux_components in add_controller commented out)

* • composition.py:  some more fixed to add_controller that now fail only one test:
    - test_agent_rep_assignement_as_controller_and_replacement

* • Passes *all* current tests

* • composition.py:
  - add_controller:  few more minor mods;
  still passes all tests

* -

* -

* -

* • controlmechanism.py:
  - __init__: resrict specification to only one of control, modulatory_signals, or control_signals (synonyms)

* -

* • composition.py:  in progress fix of bug in instantiating shadow projections for ocm.state_input_ports

* • composition.py:
  - _get_original_senders():  added support for nested composition
    needs to be checked for more than one level
    needs to be refactored to be recursive

* • optimizationcontrolmechanism.py
  - _update_state_input_ports_for_controller:  fix invalid_state_features to allow input_CIM of nested comp in agent_rep

* -

* • composition.py
 - _get_original_senders: made recursive

* • test_show_graph.py: update for fixes

* -

* • tests:  passes all in test_show_graph.py and test_report.py

* Passes all tests

* - comment clean-up

* • composition.py
 - add_controller and _get_nested_node_CIM_port:
   added support for forced assignment of NodeRole.OUTPUT for nodes specified in OCM.monitor_for_control,
   but referenced 'allow_probes' attribute still needs to be implemented

* • composition.py, optimizationcontrolmechanism.py:
  allow_probes fully implemented

* • show_graph.py:  fixed bug causing extra projections to OCM

* • composition.py:
  - _update_shadow_projections(): fix handling of deep nesting

* • optimizationcontrolmechanism.py: add agent_rep_type property

* • optimizationcontrolmechanism.py:
  - state_feature_function -> state_feature_functions

* • optimizationcontrolmechanism.py:
  - _validate_params:  validate state_feature_functions
  - _update_state_input_ports_for_controller: implement assignment of state_feature_functions

* -

* -

* • Passes all tests except test_json with 'model_with_control'

* -

* • composition.py
  - add_projection:  delete instantiation of shadow projections (handled by _update_shadow_projections)

* • composition.py
  - add_projection:  delete instantiation of shadow projections (handled by _update_shadow_projections)
  - remove calls to _update_shadows_dict

* • composition.py
  - add_projection:  delete instantiation of shadow projections (handled by _update_shadow_projections)
  - remove calls to _update_shadows_dict

* -

* • test_two_origins_two_input_ports:  crashes on failure of C->B to update

* -

* • composition.py
  - added property shadowing_dict that has shadowing ports as keys and the ports they shadow as values
  - refactored _update_shadowing_projections to use shadowing_dict

* • optimizationcontrolmechanism.py
  - _update_state_input_ports:  modified validations for nested nodes;  still failing some tests

* • optimizationcontrolmechanism.py
  - _update_state_input_ports:
     more careful and informative validation that state_input_ports are in comp or nested comp and are INPUT nodes thereof;
     passes all tests except test_two_origins_two_input_ports as before

* • composition.py
  _get_invalid_aux_components():  defer all shadow projections until _update_shadow_projections

* • composition.py
  _get_invalid_aux_components():  bug fix in test for shadow projections

* Port: _remove_projection_to_port: don't reduce variable below length 1

even ports with no incoming projections have variable at least length 1

* • composition.py
  add_node(): marked (but haven't removed) code block instantiating shadow_projections
   that seems now to be redundant with _update_shadow_projection

* • show_graph.py
  - _assign_cim_components: supress showing projections not in composition

* • composition.py:
  _analyze_graph():  add extra call to _determine_node_roles after _update_shadow_projections
  _run():  moved block of code at beginning initializing scheduler to after
           _complete_init_of_partially_initialized_nodes and _analyze_graph()

• show_graph.py
  - add test to all loops on projections:  "if proj in composition.projection"

* • show_graph.py
  - add show_projections_not_in_composition option for debugging

* • composition.py
  _update_shadow_projections(): delete unused shadow projections and corresponding ports

* • composition.py
  _update_shadow_projections(): fix bug in deletion of unused shadow projections and ports

• test_show_graph:  tests failing, need mods to accomodate changes

* • composition.py:
  _analyze_graph():  add extra call to _determine_node_roles after _update_shadow_projections
  _run():  moved block of code at beginning initializing scheduler to after
           _complete_init_of_partially_initialized_nodes and _analyze_graph()

• show_graph.py
  - add test to all loops on projections:  "if proj in composition.projection"

* • show_graph.py
  fixes; now passes all show_graph tests

* -

* • composition.py
  _update_shadow_projections:  raise error for attempt to shadow INTERNAL Node of nested comp

* -

* -

* • test_composition.py
  implemented test_shadow_nested_nodes that tests shadowing of nested nodes

* -

* -

* -

* -

* • optimizationcontrolmechanism.py: docstring mods

* • composition.py:
  - add allow_probes and exclude_probes_from_output

* • composition.py:
  - docstring mods re: allow_probes

• optimizationcontrolmechanism.py:
  - allow_probes:  eliminate DIRECT setting
  - remove _parse_monitor_for_control_input_ports (no longer needed without allow_probes=DIRECT)

* • composition.py:
  - change "exclude_probes_from_output" -> "include_probes_in_output"

* • composition.py:
  - docstring mods re: allow_probes and include_probes_in_output

* • composition.py:
  - docstring mods re: allow_probes and include_probes_in_output

* • controlmechanism.py:
  - add allow_probes handling (moved from OCM)

• optimizationcontrolmechanism.py:
  - move allow_probes to controlmechanism.py

• composition.py:
  - refactor handling of allow_probes to permit for any ControlMechanism

• objectivemechanism.py:
  - add modulatory_mechanism attribute

* • controlmechanism.py:
  - add allow_probes handling (moved from OCM)

• optimizationcontrolmechanism.py:
  - move allow_probes to controlmechanism.py

• composition.py:
  - refactor handling of allow_probes to permit for any ControlMechanism
  - add _handle_allow_probes_for_control() to reconcile setting on Composition and ControlMechanism

• objectivemechanism.py:
  - add modulatory_mechanism attribute

* • composition.py
  add assignment of learning_mechanism to objective_mechanism.modulatory_mechanism for add_learning methods

* • docstring mods

* -

* -

* • optimizationcontrolmechanism.py: docstring revs

* -

* -

* • test_composition.py:
  - add test_unnested_PROBE
  - add test_nested_PROBES
    TBD: test include_probes_in_output

* -

* • composition.py
  - add_node():  support tuple with required_role

* -

* • composition.py:
  - _determine_node_roles:
     fix bug in which nested comp was prevented from being an OUTPUT Node if,
     in addition to Nodes that qualifed as OUTPUT, it also had nodes that projected
     to Nodes in an outer comp (making it look like it was INTERNAL)

* -

* • composition.py:
  - add_node(): enforce include_probes_in_output = True for nested Compositions
  - execute():
    - replace return of output_value with get_output_value()

* -

* • CompositionInterfaceMechanism.rst:
  - correct path ref

• compositioninterfacemechanism.py:
  - docstring fixes

* • compositioninterfacemechanism.py:
  - docstring edits

* -

* -

* -

* -

* -

* -

* -

* -

* -

* • composition.py:
  add_projection:  _parse_projection_spec -> _instantiate_projection_from_spec

* • composition.py:
  add_projection:  _parse_projection_spec -> _instantiate_projection_from_spec

* • composition.py
  - add_linear_processing_pathway:
    support one->many and many->one configurations in pathway specification
    (see docstring for method)

* -

* • composition.py
  _add_linear_processing_pathway, add_projection, _is_pathway_entry_spec:
  - support many->one and one->many projections between mech and comps in pathway spec:  working
  - support set of Projections as legal entry in pathway spec:  TBI

* • composition.py
  _add_linear_processing_pathway, add_projection, _is_pathway_entry_spec:
  - support many->one and one->many projections between mech and comps in pathway spec:  working
  - support set of Projections as legal entry in pathway spec:  TBI

* • show_graph.py
  _assign_cim_components(): fix bug in which comp1_output_cim->comp2_input_cim was skipped

* • compositioninterfacemechanism.py: docstring mod

* -

* • composition.py: minor formatting

* • composition.py
  add_linear_processing_pathway(): eliminate TARGET Nodes from one->many

* -

* • composition.py:
  - add_linear_processing_pathway(): add recursive search for nested INPUT and OUTPUT nodes

* • composition.py:
  - add_linear_processing_pathway():
    - exclude Target Nodes from search for INPUT nodes (handled by learning)
    - add recursive search for nested INPUT and OUTPUT nodes

* -

* -

* • composition.py
  add_projection:  working on adding support for projection sets

* -

* -

* • composition.py
  - add _get_nested_nodes_with_same_roles_at_all_levels
  - add_linear_processing_pathway:
    call _get_nested_nodes_with_same_roles_at_all_levels

* • composition.py
  - add_linear_processing_pathway:  add support for inline specification of set of Projections

* -

* -

* • composition.py
  add_linear_processing_pathway: refactored to support inline specification of sets of Projections

* • composition.py
  add_linear_processing_pathway: docstring edits for Projection set specification

* -

* -

* -

* -

* -

* • test_composition.py:
  - add test_add_multiple_projections_for_nested_compositions - IN PROGRESS

* • test_composition.py:
  - add test_add_multiple_projections_for_nested_compositions - passes

* -

* -

* • compositioninterfacemechanism.py: add figure and example

* • composition.py:
  - get_inputs_format(): add method
  - run():  allow inputs arg to take names of nodes

* • composition.py:
  - get_inputs_format():
    - implemented num_trials arg
    - TBI: nested and labels

* -

* • composition.py
  - get_input_format(): implement show_nested_input_nodes

* • composition.py
  - get_input_format(): implement use_labels

* -

* -

* -

* -

* -

* -

* -

* -

* -

* -

* -

* • composition.py
  get_input_format() - working; TBD: tests and documentation in composition docstring

* • composition.py
  get_results_format() - working; TBD: tests and documentation in composition docstring

* -

* • composition.py: add error messages to _parse_label() and _validate_single_input()

* • composition.py: add error messages to _parse_label() and _validate_single_input()

* -

* • composition.py: docstring mods

* • composition.py - reorganize docstring section on execution

* -

* -

* -

* -

* • test_composition.py / test_input_labels_and_no_orphaning_of_nested_output_nodes:
  added tests for lables in input

* • composition.py: _validate_single_input - get rid of tests (taken care of in _parse_labels)

* • test_composition.py/test_input_labels_and_results_by_node_and_no_orphaning_of_nested_output_nodes:
  - renamed, add test for get_results_by_node

* • composition.py
  - _parse_labels:  suppress warning about targets if no learning_components in pway

* -

* • composition.py: further docstring updates for get_input_format() and get_results_by_node()

* -

* -

* -

* -

* -

* • composition.py
  - get_results_by_node():  add use_labels option

Co-authored-by: jdcpni <pniintel55>
Co-authored-by: Katherine Mantel <[email protected]>
  • Loading branch information
jdcpni and kmantel authored Dec 21, 2021
1 parent a2afe77 commit cc6242b
Show file tree
Hide file tree
Showing 9 changed files with 452 additions and 176 deletions.
2 changes: 1 addition & 1 deletion docs/source/Composition.rst
Original file line number Diff line number Diff line change
Expand Up @@ -31,4 +31,4 @@ Composition
.. automodule:: psyneulink.core.compositions.composition
:members: Composition, NodeRole, Graph
:private-members:
:exclude-members: Parameters, show_structure, CompositionError
:exclude-members: Parameters, show_structure, CompositionError, get_inputs_format
2 changes: 1 addition & 1 deletion docs/source/IntegratorFunctions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ IntegratorFunctions
.. toctree::
:maxdepth: 3

.. automodule:: psyneulink.core.components.functions.statefulfunctions.integratorfunctions
.. automodule:: psyneulink.core.components.functions.stateful.integratorfunctions
:members:
:private-members:
:exclude-members: Parameters
2 changes: 1 addition & 1 deletion docs/source/MemoryFunctions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ MemoryFunctions
.. toctree::
:maxdepth: 3

.. automodule:: psyneulink.core.components.functions.statefulfunctions.memoryfunctions
.. automodule:: psyneulink.core.components.functions.stateful.memoryfunctions
:members:
:private-members:
:exclude-members: Parameters, _validate_params, _validate, _initialize_previous_value, _function, _store_memory, _parse_memories, _is_duplicate, _get_distance, _update
2 changes: 1 addition & 1 deletion docs/source/StatefulFunction.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ StatefulFunction
.. toctree::
:maxdepth: 3

.. automodule:: psyneulink.core.components.functions.statefulfunctions.statefulfunction
.. automodule:: psyneulink.core.components.functions.stateful.statefulfunction
:members:
:private-members:
:exclude-members: Parameters
4 changes: 2 additions & 2 deletions psyneulink/core/components/mechanisms/mechanism.py
Original file line number Diff line number Diff line change
Expand Up @@ -1272,8 +1272,8 @@ class Mechanism_Base(Mechanism):
input_labels_dict : dict
contains entries that are either label:value pairs, or sub-dictionaries containing label:value pairs,
in which each label (key) specifies a string associated with a value for the InputPort(s) of the
Mechanism; see `Mechanism_Labels_Dicts` for additional details.
in which each label (key) specifies a string associated with a value for the corresponding InputPort(s)
of the Mechanism; see `Mechanism_Labels_Dicts` for additional details.
input_labels : list[str]
contains the labels corresponding to the value(s) of the InputPort(s) of the Mechanism. If the current value
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -670,7 +670,7 @@
used to further refine randomization (see `OptimizationControlMechanism_Estimation_Randomization` for additional
details).
.. _technical_note::
.. technical_note::
The *RANDOMIZATION_CONTROL_SIGNAL* ControlSignal sends a `ControlProjection` to the `ParameterPort` for the
see `Parameter` of Components specified either in the OptimizationControlMechanism's `random_variables
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -110,18 +110,19 @@
"""

import warnings
import typecheck as tc

from collections.abc import Iterable

import typecheck as tc

from psyneulink.core.components.functions.nonstateful.transferfunctions import Identity
from psyneulink.core.components.mechanisms.mechanism import Mechanism
from psyneulink.core.components.mechanisms.processing.processingmechanism import ProcessingMechanism_Base
from psyneulink.core.components.ports.inputport import InputPort
from psyneulink.core.components.ports.modulatorysignals.controlsignal import ControlSignal
from psyneulink.core.components.ports.outputport import OutputPort
from psyneulink.core.globals.context import ContextFlags, handle_external_context
from psyneulink.core.globals.keywords import COMPOSITION_INTERFACE_MECHANISM, INPUT_PORTS, OUTPUT_PORTS, PREFERENCE_SET_NAME
from psyneulink.core.globals.keywords import COMPOSITION_INTERFACE_MECHANISM, INPUT_PORTS, OUTPUT_PORTS, \
PREFERENCE_SET_NAME
from psyneulink.core.globals.parameters import Parameter
from psyneulink.core.globals.preferences.basepreferenceset import is_pref_set, REPORT_OUTPUT_PREF
from psyneulink.core.globals.preferences.preferenceset import PreferenceEntry, PreferenceLevel
Expand Down Expand Up @@ -232,27 +233,27 @@ def remove_ports(self, ports, context=None):
output_ports_marked_for_deletion.add(port)
self.user_added_ports[OUTPUT_PORTS] = self.user_added_ports[OUTPUT_PORTS] - output_ports_marked_for_deletion

def _get_destination_node_for_input_port(self, input_port, comp):
def _get_destination_node_for_input_port(self, input_port, comp=None):
"""Return Port, Node and Composition for destination of projection from input_CIM to (possibly nested) node"""
# CIM MAP ENTRIES: [RECEIVER PORT, [input_CIM InputPort, input_CIM OutputPort]]
from psyneulink.core.compositions.composition import NodeRole
# Get sender to input_port of CIM for corresponding output_port
comp = comp or self
port_map = input_port.owner.port_map
output_port = [port_map[k][1] for k in port_map if port_map[k][0] is input_port]
assert len(output_port)==1, f"PROGRAM ERROR: Expected only 1 output_port for {input_port.name} " \
f"in port_map for {input_port.owner}; found {len(output_port)}."
assert len(output_port[0].efferents)==1, f"PROGRAM ERROR: Port ({output_port.name}) expected to have " \
f"just one efferet; has {len(output_port.efferents)}."
f"just one efferent; has {len(output_port.efferents)}."
receiver = output_port[0].efferents[0].receiver
if not isinstance(receiver.owner, CompositionInterfaceMechanism):
return receiver, receiver.owner, comp
return self._get_destination_node_for_input_port(receiver, receiver.owner.composition)

def _get_source_node_for_output_port(self, output_port, comp):
def _get_source_node_for_output_port(self, output_port, comp=None):
"""Return Port, Node and Composition for source of projection to output_CIM from (possibly nested) node"""
# CIM MAP ENTRIES: [SENDER PORT, [output_CIM InputPort, output_CIM OutputPort]]
from psyneulink.core.compositions.composition import NodeRole
# Get sender to input_port of CIM for corresponding output_port
comp = comp or self
port_map = output_port.owner.port_map
input_port = [port_map[k][0] for k in port_map if port_map[k][1] is output_port]
assert len(input_port)==1, f"PROGRAM ERROR: Expected only 1 input_port for {output_port.name} " \
Expand Down
Loading

0 comments on commit cc6242b

Please sign in to comment.