Python Procedures as Subprocess Commands (xonsh.proc
)¶
Interface for running Python functions as subprocess-mode commands.
Code for several helper methods in the ProcProxy class have been reproduced without modification from subprocess.py in the Python 3.4.2 standard library. The contents of subprocess.py (and, thus, the reproduced methods) are Copyright (c) 2003-2005 by Peter Astrand <astrand@lysator.liu.se> and were licensed to the Python Software foundation under a Contributor Agreement.
-
class
xonsh.proc.
BufferedFDParallelReader
(fd, buffer=None, chunksize=1024)[source]¶ Buffered, parallel background thread reader.
Parameters: fd : int
File descriptor from which to read.
buffer : binary file-like or None, optional
A buffer to write bytes into. If None, a new BytesIO object is created.
chunksize : int, optional
The max size of the parallel reads, default 1 kb.
-
class
xonsh.proc.
CommandPipeline
(specs)[source]¶ Represents a subprocess-mode command pipeline.
Parameters: specs : list of SubprocSpec
Process specifications
Attributes
spec (SubprocSpec) The last specification in specs proc (Popen-like) The process in procs ended (bool) Boolean for if the command has stopped executing. input (str) A string of the standard input. errors (str) A string of the standard error. lines (list of str) The output lines starttime (floats or None) Pipeline start timestamp. -
end
(tee_output=True)[source]¶ End the pipeline, return the controlling terminal if needed.
Main things done in self._end().
-
itercheck
()[source]¶ Iterates through the command lines and throws an error if the returncode is non-zero.
-
tee_stdout
()[source]¶ Writes the process stdout to the output variable, line-by-line, and yields each line. This may optionally accept lines (in bytes) to iterate over, in which case it does not call iterraw().
-
alias
¶ Alias the process used.
-
args
¶ Arguments to the process.
-
attrnames
= ('stdin', 'stdout', 'stderr', 'pid', 'returncode', 'args', 'alias', 'stdin_redirect', 'stdout_redirect', 'stderr_redirect', 'timestamps', 'executed_cmd', 'input', 'output', 'errors')¶
-
err
¶ Error messages as a string.
-
executed_cmd
¶ The resolve and executed command.
-
inp
¶ Creates normalized input string from args.
-
nonblocking
= (<class '_io.BytesIO'>, <class 'xonsh.proc.NonBlockingFDReader'>, <class 'xonsh.proc.ConsoleParallelReader'>)¶
-
out
¶ Output value as a str.
-
output
¶ Non-blocking, lazy access to output
-
pid
¶ Process identifier.
-
returncode
¶ Process return code, waits until command is completed.
-
rtn
¶ Alias to return code.
-
stderr
¶ Process stderr.
-
stderr_postfix
¶ Postfix to print after stderr, as bytes.
-
stderr_prefix
¶ Prefix to print in front of stderr, as bytes.
-
stderr_redirect
¶ Redirection used for stderr.
-
stdin
¶ Process stdin.
-
stdin_redirect
¶ Redirection used for stdin.
-
stdout
¶ Process stdout.
-
stdout_redirect
¶ Redirection used for stdout.
-
timestamps
¶ The start and end time stamps.
-
-
class
xonsh.proc.
ConsoleParallelReader
(fd, buffer=None, chunksize=1024, timeout=None)[source]¶ Parallel reader for consoles that runs in a background thread. This is only needed, available, and useful on Windows.
Parameters: fd : int
Standard buffer file descriptor, 0 for stdin, 1 for stdout (default), and 2 for stderr.
buffer : ctypes.c_wchar_p, optional
An existing buffer to (re-)use.
chunksize : int, optional
The max size of the parallel reads, default 1 kb.
timeout : float, optional
The queue reading timeout.
-
class
xonsh.proc.
FileThreadDispatcher
(default=None)[source]¶ Dispatches to different file handles depending on the current thread. Useful if you want file operation to go to different places for different threads.
Parameters: default : file-like or None, optional
The file handle to write to if a thread cannot be found in the registry. If None, a new in-memory instance.
Attributes
registry (dict) Maps thread idents to file handles. -
register
(handle)[source]¶ Registers a file handle for the current thread. Returns self so that this method can be used in a with-statement.
-
write
(s)[source]¶ Writes to this thread’s handle. This also flushes, just to be extra sure the string was written.
-
available
¶ True if the thread is available in the registry.
-
buffer
¶ Gets the buffer for this thread’s handle.
-
closed
¶ Is the thread’s handle closed.
-
encoding
¶ Gets the encoding for this thread’s handle.
-
errors
¶ Gets the errors for this thread’s handle.
-
handle
¶ Gets the current handle for the thread.
-
line_buffering
¶ Gets if line buffering for this thread’s handle enabled.
-
newlines
¶ Gets the newlines for this thread’s handle.
-
-
class
xonsh.proc.
HiddenCommandPipeline
(specs)[source]¶ Parameters: specs : list of SubprocSpec
Process specifications
Attributes
spec (SubprocSpec) The last specification in specs proc (Popen-like) The process in procs ended (bool) Boolean for if the command has stopped executing. input (str) A string of the standard input. errors (str) A string of the standard error. lines (list of str) The output lines starttime (floats or None) Pipeline start timestamp.
-
class
xonsh.proc.
NonBlockingFDReader
(fd, timeout=None)[source]¶ A class for reading characters from a file descriptor on a background thread. This has the advantages that the calling thread can close the file and that the reading does not block the calling thread.
Parameters: fd : int
A file descriptor
timeout : float or None, optional
The queue reading timeout.
-
class
xonsh.proc.
PopenThread
(*args, stdin=None, stdout=None, stderr=None, **kwargs)[source]¶ A thread for running and managing subprocess. This allows reading from the stdin, stdout, and stderr streams in a non-blocking fashion.
This takes the same arguments and keyword arguments as regular Popen. This requires that the captured_stdout and captured_stderr attributes to be set following instantiation.
-
run
()[source]¶ Runs the subprocess by performing a parallel read on stdin if allowed, and copying bytes from captured_stdout to stdout and bytes from captured_stderr to stderr.
-
wait
(timeout=None)[source]¶ Dispatches to Popen.wait(), but also does process cleanup such as joining this thread and replacing the original window size signal handler.
-
returncode
¶ Process return code.
-
signal
¶ Process signal, or None.
-
-
class
xonsh.proc.
PrevProcCloser
(pipeline)[source]¶ Previous process closer thread for pipelines whose last command is itself unthreadable. This makes sure that the pipeline is driven forward and does not deadlock.
Parameters: pipeline : CommandPipeline
The pipeline whose prev procs we should close.
-
class
xonsh.proc.
ProcProxy
(f, args, stdin=None, stdout=None, stderr=None, universal_newlines=False, close_fds=False, env=None)[source]¶ This is process proxy class that runs its alias functions on the same thread that it was called from, which is typically the main thread. This prevents the process from running on a background thread, but enables debugger and profiler tools (functions) be run on the same thread that they are attempting to debug.
-
class
xonsh.proc.
ProcProxyThread
(f, args, stdin=None, stdout=None, stderr=None, universal_newlines=False, close_fds=False, env=None)[source]¶ Class representing a function to be run as a subprocess-mode command.
Parameters: f : function
The function to be executed.
args : list
A (possibly empty) list containing the arguments that were given on the command line
stdin : file-like, optional
A file-like object representing stdin (input can be read from here). If stdin is not provided or if it is explicitly set to None, then an instance of io.StringIO representing an empty file is used.
stdout : file-like, optional
A file-like object representing stdout (normal output can be written here). If stdout is not provided or if it is explicitly set to None, then sys.stdout is used.
stderr : file-like, optional
A file-like object representing stderr (error output can be written here). If stderr is not provided or if it is explicitly set to None, then sys.stderr is used.
universal_newlines : bool, optional
Whether or not to use universal newlines.
close_fds : bool, optional
Whether or not to close file descriptors. This is here for Popen compatability and currently does nothing.
env : Mapping, optional
Environment mapping.
-
poll
()[source]¶ Check if the function has completed.
Returns: None if the function is still executing, and the returncode otherwise
-
-
class
xonsh.proc.
QueueReader
(fd, timeout=None)[source]¶ Provides a file-like interface to reading from a queue.
Parameters: fd : int
A file descriptor
timeout : float or None, optional
The queue reading timeout.
-
xonsh.proc.
parse_proxy_return
(r, stdout, stderr)[source]¶ Proxies may return a variety of outputs. This handles them generally.
Parameters: r : tuple, str, int, or None
Return from proxy function
stdout : file-like
Current stdout stream
stdout : file-like
Current stderr stream
Returns: cmd_result : int
The return code of the proxy
-
xonsh.proc.
partial_proxy
(f)[source]¶ Dispatches the appropriate proxy function based on the number of args.
-
xonsh.proc.
pause_call_resume
(p, f, *args, **kwargs)[source]¶ For a process p, this will call a function f with the remaining args and and kwargs. If the process cannot accept signals, the function will be called.
Parameters: p : Popen object or similar
f : callable
args : remaining arguments
kwargs : keyword arguments
-
xonsh.proc.
populate_buffer
(reader, fd, buffer, chunksize)[source]¶ Reads bytes from the file descriptor and copies them into a buffer.
The reads happen in parallel using the pread() syscall; which is only available on POSIX systems. If the read fails for any reason, the reader is flagged as closed.
-
xonsh.proc.
populate_console
(reader, fd, buffer, chunksize, queue, expandsize=None)[source]¶ Reads bytes from the file descriptor and puts lines into the queue. The reads happened in parallel, using xonsh.winutils.read_console_output_character(), and is thus only available on windows. If the read fails for any reason, the reader is flagged as closed.
-
xonsh.proc.
populate_fd_queue
(reader, fd, queue)[source]¶ Reads 1 kb of data from a file descriptor into a queue. If this ends or fails, it flags the calling reader object as closed.
-
xonsh.proc.
proxy_five
(f, args, stdin, stdout, stderr, spec, stack)[source]¶ Calls a proxy function which takes four parameter: args, stdin, stdout, stderr, and spec.
-
xonsh.proc.
proxy_four
(f, args, stdin, stdout, stderr, spec, stack)[source]¶ Calls a proxy function which takes four parameter: args, stdin, stdout, and stderr.
-
xonsh.proc.
proxy_one
(f, args, stdin, stdout, stderr, spec, stack)[source]¶ Calls a proxy function which takes one parameter: args
-
xonsh.proc.
proxy_three
(f, args, stdin, stdout, stderr, spec, stack)[source]¶ Calls a proxy function which takes three parameter: args, stdin, stdout.
-
xonsh.proc.
proxy_two
(f, args, stdin, stdout, stderr, spec, stack)[source]¶ Calls a proxy function which takes two parameter: args and stdin.
-
xonsh.proc.
proxy_zero
(f, args, stdin, stdout, stderr, spec, stack)[source]¶ Calls a proxy function which takes no parameters.
-
xonsh.proc.
safe_fdclose
(handle, cache=None)[source]¶ Closes a file handle in the safest way possible, and potentially storing the result.
-
xonsh.proc.
safe_flush
(handle)[source]¶ Attempts to safely flush a file handle, returns success bool.
-
xonsh.proc.
safe_readable
(handle)[source]¶ Attempts to find if the handle is readable without throwing an error.
-
xonsh.proc.
safe_readlines
(handle, hint=-1)[source]¶ Attempts to read lines without throwing an error.