Python Logging Module¶
You can also control the verbosity of Python's logging messages by setting the logging level to ERROR, WARNING, INFO, or DEBUG. This can be configured independently for different modules.
Let's turn on INFO level messages for the optimization module:
In [2]:
Copied!
import pypsa
network = pypsa.examples.ac_dc_meshed()
import pypsa
network = pypsa.examples.ac_dc_meshed()
/home/runner/work/PyPSA/PyPSA/pypsa/network/io.py:2082: FutureWarning: pandas infers the `str` dtype for string data since its version 3.0. PyPSA still converts it back to numpy object dtype on import, but will keep it from PyPSA 2.0 on. Set `pypsa.options.api.legacy_string_dtype` explicitly to suppress this warning. new_static = _coerce_string_dtypes(new_static)
INFO:pypsa.network.io:Imported network 'AC-DC-Meshed' has buses, carriers, generators, global_constraints, lines, links, loads
In [3]:
Copied!
import logging
pypsa.optimization.optimize.logger.setLevel(logging.INFO)
network.optimize()
import logging
pypsa.optimization.optimize.logger.setLevel(logging.INFO)
network.optimize()
/tmp/ipykernel_8583/1123755349.py:5: FutureWarning: The default value of `include_objective_constant` will change from True to False in version 2.0. Set `include_objective_constant` explicitly to suppress this warning. Using False improves LP numerical conditioning by not including the objective constant as a variable. network.optimize() WARNING:pypsa.consistency:The following lines have zero x, which could break the linear load flow: Index(['2', '3', '4'], dtype='object', name='name')
WARNING:pypsa.consistency:The following lines have zero r, which could break the linear load flow: Index(['0', '1', '5', '6'], dtype='object', name='name')
INFO:linopy.model: Solve problem using Gurobi solver
INFO:linopy.model:Solver options: - log_to_console: False
INFO:linopy.io: Writing time: 0.06s
Set parameter WLSAccessID
Set parameter WLSSecret
Set parameter LicenseID to value 2537914
Academic license 2537914 - for non-commercial use only - registered to l.___@tu-berlin.de
Read LP format model from file /tmp/linopy-problem-3341939e.lp
Reading time = 0.00 seconds
obj: 468 rows, 188 columns, 1007 nonzeros
Set parameter LogToConsole to value 0
INFO:linopy.constants: Optimization successful: Status: ok Termination condition: optimal Solution: 188 primals, 468 duals Objective: -3.47e+06 Solver: gurobi Runtime: 0.00s Dual bound: -3.47e+06 Solver model: available Solver message: 2
INFO:pypsa.optimization.optimize:The shadow-prices of the constraints Generator-ext-p-lower, Generator-ext-p-upper, Line-ext-s-lower, Line-ext-s-upper, Link-ext-p-lower, Link-ext-p-upper, Kirchhoff-Voltage-Law were not assigned to the network.
Out[3]:
('ok', 'optimal')
Now let's reduce verbosity by setting the optimization module to only show WARNING level messages.
In [4]:
Copied!
pypsa.optimization.optimize.logger.setLevel(logging.WARNING)
network.optimize()
pypsa.optimization.optimize.logger.setLevel(logging.WARNING)
network.optimize()
/tmp/ipykernel_8583/1121149820.py:3: FutureWarning: The default value of `include_objective_constant` will change from True to False in version 2.0. Set `include_objective_constant` explicitly to suppress this warning. Using False improves LP numerical conditioning by not including the objective constant as a variable. network.optimize()
WARNING:pypsa.consistency:The following lines have zero x, which could break the linear load flow: Index(['2', '3', '4'], dtype='object', name='name')
WARNING:pypsa.consistency:The following lines have zero r, which could break the linear load flow: Index(['0', '1', '5', '6'], dtype='object', name='name')
INFO:linopy.model: Solve problem using Gurobi solver
INFO:linopy.model:Solver options: - log_to_console: False
INFO:linopy.io: Writing time: 0.06s
Set parameter WLSAccessID
Set parameter WLSSecret
Set parameter LicenseID to value 2537914
Academic license 2537914 - for non-commercial use only - registered to l.___@tu-berlin.de
Read LP format model from file /tmp/linopy-problem-d52d6ip2.lp
Reading time = 0.00 seconds
obj: 468 rows, 188 columns, 1007 nonzeros
Set parameter LogToConsole to value 0
INFO:linopy.constants: Optimization successful: Status: ok Termination condition: optimal Solution: 188 primals, 468 duals Objective: -3.47e+06 Solver: gurobi Runtime: 0.00s Dual bound: -3.47e+06 Solver model: available Solver message: 2
Out[4]:
('ok', 'optimal')
Now let's turn on DEBUG level messages for the power flow module to see the most detailed output.
In [5]:
Copied!
pypsa.network.power_flow.logger.setLevel(logging.DEBUG)
out = network.lpf()
pypsa.network.power_flow.logger.setLevel(logging.DEBUG)
out = network.lpf()
DEBUG:pypsa.network.power_flow:Slack bus for sub-network 0 is Manchester
DEBUG:pypsa.network.power_flow:Slack bus for sub-network 1 is Norwich DC
DEBUG:pypsa.network.power_flow:Slack bus for sub-network 2 is Frankfurt
DEBUG:pypsa.network.power_flow:No slack generator found in sub-network 3, using Norway Wind as the slack generator
DEBUG:pypsa.network.power_flow:Slack bus for sub-network 3 is Norway
DEBUG:pypsa.network.power_flow:Slack bus for sub-network 0 is Manchester
INFO:pypsa.network.power_flow:Performing linear load-flow on AC sub-network <pypsa.SubNetwork object at 0x7fb51f90dd00> for snapshot(s) DatetimeIndex(['2015-01-01 00:00:00', '2015-01-01 01:00:00',
'2015-01-01 02:00:00', '2015-01-01 03:00:00',
'2015-01-01 04:00:00', '2015-01-01 05:00:00',
'2015-01-01 06:00:00', '2015-01-01 07:00:00',
'2015-01-01 08:00:00', '2015-01-01 09:00:00'],
dtype='datetime64[ns]', name='snapshot', freq=None)
DEBUG:pypsa.network.power_flow:Slack bus for sub-network 1 is Norwich DC
INFO:pypsa.network.power_flow:Performing linear load-flow on DC sub-network <pypsa.SubNetwork object at 0x7fb51f90de10> for snapshot(s) DatetimeIndex(['2015-01-01 00:00:00', '2015-01-01 01:00:00',
'2015-01-01 02:00:00', '2015-01-01 03:00:00',
'2015-01-01 04:00:00', '2015-01-01 05:00:00',
'2015-01-01 06:00:00', '2015-01-01 07:00:00',
'2015-01-01 08:00:00', '2015-01-01 09:00:00'],
dtype='datetime64[ns]', name='snapshot', freq=None)
DEBUG:pypsa.network.power_flow:Slack bus for sub-network 2 is Frankfurt
INFO:pypsa.network.power_flow:Performing linear load-flow on AC sub-network <pypsa.SubNetwork object at 0x7fb5242aaa50> for snapshot(s) DatetimeIndex(['2015-01-01 00:00:00', '2015-01-01 01:00:00',
'2015-01-01 02:00:00', '2015-01-01 03:00:00',
'2015-01-01 04:00:00', '2015-01-01 05:00:00',
'2015-01-01 06:00:00', '2015-01-01 07:00:00',
'2015-01-01 08:00:00', '2015-01-01 09:00:00'],
dtype='datetime64[ns]', name='snapshot', freq=None)
DEBUG:pypsa.network.power_flow:Slack bus for sub-network 3 is Norway
INFO:pypsa.network.power_flow:Performing linear load-flow on AC sub-network <pypsa.SubNetwork object at 0x7fb5242aae50> for snapshot(s) DatetimeIndex(['2015-01-01 00:00:00', '2015-01-01 01:00:00',
'2015-01-01 02:00:00', '2015-01-01 03:00:00',
'2015-01-01 04:00:00', '2015-01-01 05:00:00',
'2015-01-01 06:00:00', '2015-01-01 07:00:00',
'2015-01-01 08:00:00', '2015-01-01 09:00:00'],
dtype='datetime64[ns]', name='snapshot', freq=None)
Finally, let's turn off messages for the power flow module by setting it back to ERROR level.
In [6]:
Copied!
pypsa.network.power_flow.logger.setLevel(logging.ERROR)
out = network.lpf()
pypsa.network.power_flow.logger.setLevel(logging.ERROR)
out = network.lpf()
In [8]:
Copied!
# Access a deprecated feature to see a warning
network.components.generators.component_names
# Access a deprecated feature to see a warning
network.components.generators.component_names
/tmp/ipykernel_8583/1732202595.py:2: DeprecationWarning: c.component_names is deprecated, use c.names instead network.components.generators.component_names
Out[8]:
Index(['Manchester Wind', 'Manchester Gas', 'Norway Wind', 'Norway Gas',
'Frankfurt Wind', 'Frankfurt Gas'],
dtype='object', name='name')
In [9]:
Copied!
import warnings
warnings.filterwarnings("ignore", category=DeprecationWarning)
network.components.generators.component_names
import warnings
warnings.filterwarnings("ignore", category=DeprecationWarning)
network.components.generators.component_names
Out[9]:
Index(['Manchester Wind', 'Manchester Gas', 'Norway Wind', 'Norway Gas',
'Frankfurt Wind', 'Frankfurt Gas'],
dtype='object', name='name')
Solver Output¶
The log_to_console parameter controls whether the solver (e.g., HiGHS, Gurobi) prints its progress to the console. By default, pypsa.options.params.optimize.log_to_console is True, but in these documentation examples it's set to False for cleaner output.
Let's see the difference:
In [10]:
Copied!
# With log_to_console=False (default in docs): only Python logging messages
network.optimize(log_to_console=False)
# With log_to_console=False (default in docs): only Python logging messages
network.optimize(log_to_console=False)
/tmp/ipykernel_8583/733640962.py:2: FutureWarning: The default value of `include_objective_constant` will change from True to False in version 2.0. Set `include_objective_constant` explicitly to suppress this warning. Using False improves LP numerical conditioning by not including the objective constant as a variable. network.optimize(log_to_console=False) WARNING:pypsa.consistency:Encountered nan's in varying data 'p0' for columns ['Norwich Converter', 'Norway Converter', 'Bremen Converter', 'DC link'] of component 'Link'.
WARNING:pypsa.consistency:Encountered nan's in varying data 'p1' for columns ['Norwich Converter', 'Norway Converter', 'Bremen Converter', 'DC link'] of component 'Link'.
WARNING:pypsa.consistency:The following lines have zero x, which could break the linear load flow: Index(['2', '3', '4'], dtype='object', name='name')
WARNING:pypsa.consistency:The following lines have zero r, which could break the linear load flow: Index(['0', '1', '5', '6'], dtype='object', name='name')
INFO:linopy.model: Solve problem using Gurobi solver
INFO:linopy.model:Solver options: - log_to_console: False
INFO:linopy.io: Writing time: 0.06s
Set parameter WLSAccessID
Set parameter WLSSecret
Set parameter LicenseID to value 2537914
Academic license 2537914 - for non-commercial use only - registered to l.___@tu-berlin.de
Read LP format model from file /tmp/linopy-problem-p9tzg2a_.lp
Reading time = 0.00 seconds
obj: 468 rows, 188 columns, 1007 nonzeros
Set parameter LogToConsole to value 0
INFO:linopy.constants: Optimization successful: Status: ok Termination condition: optimal Solution: 188 primals, 468 duals Objective: -3.47e+06 Solver: gurobi Runtime: 0.00s Dual bound: -3.47e+06 Solver model: available Solver message: 2
Out[10]:
('ok', 'optimal')
In [11]:
Copied!
# With log_to_console=True (default setting): verbose solver output
network.optimize(log_to_console=True)
# With log_to_console=True (default setting): verbose solver output
network.optimize(log_to_console=True)
/tmp/ipykernel_8583/2160006920.py:2: FutureWarning: The default value of `include_objective_constant` will change from True to False in version 2.0. Set `include_objective_constant` explicitly to suppress this warning. Using False improves LP numerical conditioning by not including the objective constant as a variable. network.optimize(log_to_console=True)
WARNING:pypsa.consistency:The following lines have zero x, which could break the linear load flow: Index(['2', '3', '4'], dtype='object', name='name')
WARNING:pypsa.consistency:The following lines have zero r, which could break the linear load flow: Index(['0', '1', '5', '6'], dtype='object', name='name')
INFO:linopy.model: Solve problem using Gurobi solver
INFO:linopy.model:Solver options: - log_to_console: True
INFO:linopy.io: Writing time: 0.06s
Set parameter WLSAccessID
Set parameter WLSSecret
Set parameter LicenseID to value 2537914
Academic license 2537914 - for non-commercial use only - registered to l.___@tu-berlin.de
Read LP format model from file /tmp/linopy-problem-fhgvwdep.lp
Reading time = 0.00 seconds
obj: 468 rows, 188 columns, 1007 nonzeros
Set parameter LogToConsole to value 1
Gurobi Optimizer version 13.0.2 build v13.0.2rc1 (linux64 - "Ubuntu 24.04.4 LTS")
CPU model: AMD EPYC 7763 64-Core Processor, instruction set [SSE2|AVX|AVX2]
Thread count: 2 physical cores, 4 logical processors, using up to 4 threads
Academic license 2537914 - for non-commercial use only - registered to l.___@tu-berlin.de
Optimize a model with 468 rows, 188 columns and 1007 nonzeros (Min)
Model fingerprint: 0x8e41a0d0
Model has 78 linear objective coefficients
Coefficient statistics:
Matrix range [1e-02, 1e+00]
Objective range [9e-03, 3e+03]
Bounds range [2e+07, 2e+07]
RHS range [9e-01, 1e+03]
Presolve removed 190 rows and 71 columns
Presolve time: 0.01s
Presolved: 278 rows, 117 columns, 1291 nonzeros
Iteration Objective Primal Inf. Dual Inf. Time
0 -2.1204529e+07 2.152701e+04 0.000000e+00 0s
172 -3.4742560e+06 0.000000e+00 0.000000e+00 0s
Solved in 172 iterations and 0.01 seconds (0.00 work units)
Optimal objective -3.474256041e+06
INFO:linopy.constants: Optimization successful: Status: ok Termination condition: optimal Solution: 188 primals, 468 duals Objective: -3.47e+06 Solver: gurobi Runtime: 0.01s Dual bound: -3.47e+06 Solver model: available Solver message: 2
Out[11]:
('ok', 'optimal')